Ray Data Ingest into AIR Trainers

See also

See this AIR Data ingest guide for usage examples.

air.session.get_dataset_shard([dataset_name])	Returns the ray.data.DataIterator shard for this worker.
DataIterator()	An iterator for reading records from a Dataset or DatasetPipeline.
ray.train.DataConfig([datasets_to_split, ...])	Class responsible for configuring Train dataset preprocessing.

Debugging Utilities

make_local_dataset_iterator(dataset, ...)	A helper function to create a local DataIterator, like the one returned by get_dataset_shard().
DummyTrainer(*args, **kwargs)	A Trainer that does nothing except read the data for a given number of epochs.