PyRoboFrames

High-performance dataloader for robot learning. LeRobot format support, hardware-accelerated video decode via VideoToolbox, zero-copy MLX arrays. Optimized for Apple Silicon and edge robotics.

---

Product Overview

PyRoboFrames is a proprietary, production-grade dataloader for robot learning. Hardware-accelerated video decode with zero-copy data pipelines optimized for Apple Silicon and edge devices.

Why Robot Learning Teams Choose This

The Problem:

• Loading robot video datasets is slow (CPU decode bottleneck)
• Memory usage explodes with large MCAP files
• Data pipeline complexity delays model iteration

The Solution:

• VideoToolbox hardware acceleration (5-10x faster)
• Zero-copy MLX array integration
• Streaming data loading (never load full dataset)
• LeRobot format native support

Result: 5-10x faster data loading, 50% less memory, iterate on models 10x faster.

---

Installation

pip install pyroboframes
# or with uv
uv pip install pyroboframes

# Verify installation
roboframes --version

Requirements

Runtime

• Python 3.10+
• macOS (Apple Silicon M1/M2/M3) or Linux (x86_64)
• 4GB+ RAM (recommended 8GB+ for large datasets)

Dependencies

• numpy ≥1.20.0
• MLX ≥0.1.0 (for GPU arrays, optional on macOS)
• PyTorch ≥2.0.0 (optional, for model training)
• PyArrow (for Parquet support)

Hardware

• Apple Silicon (M1/M2/M3) strongly recommended for VideoToolbox acceleration
• Linux with GPU support (CUDA/ROCm) optional

Distribution Model

Proprietary-first distribution:

• ✅ Wheels-only via PyPI (no source code)
• ✅ Hardware-accelerated on Apple Silicon
• ✅ 142 comprehensive tests
• ✅ Used in production robotics labs

---

Quick Start

from pyroboframes import LeRobotDataset

# Load LeRobot dataset with hardware acceleration
dataset = LeRobotDataset('lerobot/pusht')

# Stream data as MLX arrays (zero-copy)
for batch in dataset.iter_mlx(batch_size=32):
    images = batch['images']  # MLX array, GPU-ready
    actions = batch['actions']
    
    # Your model training code
    model.train(images, actions)

# Total memory: only one batch at a time

---

Features

• Hardware Acceleration: VideoToolbox for video decode
• Zero-Copy Integration: Direct to MLX arrays
• LeRobot Support: Native MCAP and HF Hub format
• Streaming: Process massive datasets without loading into memory
• Apple Silicon Optimized: Native support for M1/M2/M3
• Multi-format: Video, sensor streams, action sequences
• Production Ready: 142 tests, observability included

---

Performance

• Video decode: 5-10x faster than CPU (hardware-accelerated)
• Memory usage: Constant regardless of dataset size
• Throughput: 1000+ samples/sec on Apple Silicon

---

Detailed Features

Data Loading & Formats

• LeRobot format (MCAP, HF Hub) native support
• Video formats: MP4, MOV, AVI (hardware-decoded)
• Sensor streams: IMU, joint states, proprioception
• Action sequences: continuous/discrete
• Batch processing with configurable sizes
• Lazy loading (data loaded on-demand)

Hardware Acceleration

• VideoToolbox (macOS) for video decode
• MLX integration for GPU-ready arrays
• Zero-copy data transfer (no unnecessary copies)
• Multi-threaded frame fetching
• Prefetching for smooth pipeline

Data Pipeline Optimization

• Streaming mode (process massive datasets)
• Caching layer (memory-efficient)
• Shuffling and augmentation support
• Async data loading (non-blocking)
• Jitter correction for sensor data

ML Integration

• Direct MLX array output (GPU-ready)
• PyTorch compatibility (convert as needed)
• NumPy compatibility (CPU fallback)
• Configurable data types (float32, float16, etc.)

---

Examples

Basic LeRobot Loading

from pyroboframes import LeRobotDataset

# Load dataset from Hugging Face Hub
dataset = LeRobotDataset('lerobot/pusht')

# Iterate with automatic hardware acceleration
for batch in dataset.iter_mlx(batch_size=32):
    images = batch['images']    # Shape: (32, 3, 224, 224)
    actions = batch['actions']  # Shape: (32, action_dim)
    print(f"Batch loaded: {images.shape}")

Training Loop Integration

from pyroboframes import LeRobotDataset
import mlx.core as mx

dataset = LeRobotDataset('lerobot/aloha', split='train')

# Zero-copy MLX arrays for GPU training
for epoch in range(10):
    for batch in dataset.iter_mlx(batch_size=64, shuffle=True):
        images = batch['images']
        actions = batch['actions']
        
        # MLX arrays stay on GPU (no CPU copy)
        loss = model.forward(images, actions)
        optimizer.update(model.parameters(), grads)

print(f"Training complete. Memory used: constant")

Sensor Data & Multi-Modal

dataset = LeRobotDataset('lerobot/pusht')

for sample in dataset:
    # Video frame (hardware-accelerated)
    frame = sample['observation.image']  # RGB, shape (H, W, 3)
    
    # Sensor streams
    joint_positions = sample['observation.state']
    proprioception = sample['observation.imu']
    
    # Action target
    action = sample['action']
    
    # Process all modalities
    process_multimodal(frame, joint_positions, proprioception, action)

Large Dataset Streaming

from pyroboframes import LeRobotDataset

# Dataset larger than available RAM
dataset = LeRobotDataset('lerobot/large_dataset')

# Stream without loading full dataset
batch_count = 0
for batch in dataset.iter_mlx(batch_size=128, stream=True):
    batch_count += 1
    model.train_step(batch)
    
    if batch_count % 1000 == 0:
        print(f"Processed {batch_count} batches. Memory: constant")

# Memory never exceeds one batch size

Data Caching & Prefetch

dataset = LeRobotDataset('lerobot/pusht')

# Enable prefetching for smoother training
loader = dataset.iter_mlx(
    batch_size=32,
    prefetch_batches=2,  # Load next 2 batches while training
    cache_frames=False    # Video decode on-the-fly
)

for batch in loader:
    model.train(batch)  # No waiting for data

---

API Reference

LeRobotDataset Class

LeRobotDataset(
    dataset_id: str,
    split: str = "train",
    cache_dir: str = None,
    format: str = "auto"  # auto, hf, mcap
)

Methods

• iter_mlx(batch_size, shuffle=False, stream=False, prefetch_batches=1)

  • Iterate with MLX arrays (GPU-ready)
  • batch_size: Number of samples per batch
  • shuffle: Random order per epoch
  • stream: Process without full load
  • prefetch_batches: Async batches to load

• iter_numpy(batch_size)

  • Iterate with NumPy arrays (CPU)

• iter_torch(batch_size)

  • Iterate with PyTorch tensors

• get_sample(idx)

  • Get single sample by index

• info()

  • Dataset statistics and metadata

Sample Structure

{
    'observation.image': MLXArray,      # Video frame
    'observation.state': MLXArray,       # Joint positions
    'observation.imu': MLXArray,         # IMU data
    'action': MLXArray,                  # Action target
    'episode_index': int,
    'frame_index': int,
}

---

Quality & Testing

• 142 tests passing
• Production-grade — used in robotics research labs
• Observability — performance profiling included

---

Support

For production deployments: mullassery@gmail.com

---

Version: 1.2.2
License: Proprietary
Distribution: Wheels-only via PyPI
Python: 3.10+

Built for high-performance robot learning.