Grain 0.2.16

• New features:

  • Adds support for filtering Grain-internal stack frames from user-thrown
    errors.
  • Adds experimental support for get_next_index and set_next_index to fetch
    and advance a grain.DatasetIterator to the given produced element index.
  • Switches to multithreading instead of multiprocessing in
    IterDataset.mp_prefetch when free-threaded Python is detected.
  • grain.DataLoaderIterator can now asynchronously start processing elements
    in background with start_prefetch call.
  • Immediately starts processing elements in the background after restoring
    from a checkpoint with Orbax.
  • Exposes SharedMemoryArrayMetadata in a public API as a metadata descriptor for SharedMemoryArray.

• Breaking changes:

  • Custom implementations of RandomAccessDataSource should accept int
    index in __getitem__. Legacy paths that handle SupportsIndex will still
    work at runtime, but depending on the type checker in use, if you're
    directly inheriting from grain.RandomAccessDataSource and call
    super().__getitem__ with Supportsindex you may see a type checking
    error. Switch to int to fix it.

• Deprecations:

  • Deprecates grain.python.experimental.MultiprocessPrefetchIterDataset,
    use the graduated version instead: grain.IterDataset.mp_prefetch.
  • Deprecates grain.python.experimental.ConcatenateMapDataset, use the
    graduated version instead: grain.MapDataset.concatenate.

• Bug fixes:

  • Fixes bug in WindowShuffleIterDataset checkpointing.