User Guides

If you’re new to Ray Data, we recommend starting with the Ray Data Key Concepts. This user guide will help you navigate the Ray Data project and show you how achieve several tasks.

• Loading Data
  • Reading files
  • Loading data from other libraries
  • Reading databases
  • Creating synthetic data
  • Loading other data sources
  • Performance considerations
• Transforming Data
  • Transforming rows
  • Transforming batches
  • Groupby and transforming groups
  • Shuffling rows
  • Repartitioning data
• Inspecting Data
  • Describing datasets
  • Inspecting rows
  • Inspecting batches
  • Inspecting execution statistics
• Iterating over Data
  • Iterating over rows
  • Iterating over batches
  • Iterating over batches with shuffling
  • Splitting datasets for distributed parallel training
• Saving Data
  • Writing data to files
  • Converting Datasets to other Python libraries
• Working with Images
  • Reading images
  • Transforming images
  • Performing inference on images
  • Saving images
• Working with Text
  • Reading text files
  • Transforming text
  • Performing inference on text
  • Saving text
• Working with Tensors
  • Tensor data representation
  • Transforming tensor data
  • Saving tensor data
• Working with PyTorch
  • Iterating over torch tensors for training
  • Transformations with torch tensors
  • Batch inference with PyTorch
  • Saving Datasets containing torch tensors
  • Migrating from PyTorch Datasets and DataLoaders
• End-to-end: Offline Batch Inference
  • Quickstart
  • More examples
  • Configuration and troubleshooting
  • Using models from Ray Train
  • Benchmarks
• Advanced: Performance Tips and Tuning
  • Optimizing transforms
  • Optimizing reads
  • Optimizing shuffles
  • Configuring execution
  • Reproducibility
  • Monitoring your application
• Using Preprocessors
  • Overview
  • Life of an AIR preprocessor
  • Types of preprocessors
  • Which preprocessor should you use?