-
Notifications
You must be signed in to change notification settings - Fork 7.3k
Open
Labels
community-backlogdataRay Data-related issuesRay Data-related issuesenhancementRequest for new feature and/or capabilityRequest for new feature and/or capabilitytrainRay Train Related IssueRay Train Related IssuetriageNeeds triage (eg: priority, bug/not-bug, and owning component)Needs triage (eg: priority, bug/not-bug, and owning component)usability
Description
Description
Having a row hash column will be useful for checkpointing, e.g., in Ray Data pipeline or Ray Train use cases
Use case
Ray Data (read-map-write): #59409
requires an existing "id" column. This PR will automatically generate a row_hash column that simplifies that.
Ray Train:
- many users are implementing their own stateful dataloader (as indicated in Ray Summit..)
- Anyscale has their own mid-epoch resumption: https://docs.anyscale.com/runtime/mid-epoch-resumption
- and
generate_id_columnoption.
- and
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
community-backlogdataRay Data-related issuesRay Data-related issuesenhancementRequest for new feature and/or capabilityRequest for new feature and/or capabilitytrainRay Train Related IssueRay Train Related IssuetriageNeeds triage (eg: priority, bug/not-bug, and owning component)Needs triage (eg: priority, bug/not-bug, and owning component)usability
Type
Projects
Status
In progress