turboloader 0.4.0

High-performance data loading for machine learning with cloud storage support and GPU acceleration

TurboLoader

High-Performance ML Data Loading Library

---

Overview

TurboLoader is a high-performance data loading library designed to accelerate ML training by replacing Python's multiprocessing-based data loaders with efficient C++ native threads and thread-safe concurrent data structures.

Key Features:

• 🚀 Native C++ Implementation with Python bindings via pybind11
• ⚡ SIMD-Optimized Transforms using AVX2/AVX-512/NEON
• 🔒 Thread-Safe Concurrent Queues for reliable multi-threaded data passing
• 🧵 C++ Native Threads (no Python GIL, no multiprocessing overhead)
• 💾 Zero-Copy Memory-Mapped I/O for efficient file reading
• 📦 WebDataset TAR Format support for sharded datasets
• 🎯 SIMD-Accelerated Image Decoders (JPEG, PNG, WebP)
• 🎨 7 Data Augmentation Transforms with SIMD optimization
• 🐍 PyTorch-Compatible API drop-in replacement

---

Performance

Current Status (v0.3.8)

IMPORTANT: TurboLoader v0.3.8 has a critical performance issue and is currently 2.7x slower than PyTorch DataLoader. We have identified the root causes and are implementing a complete rewrite in TurboLoader (see roadmap below).

End-to-End Data Loading Benchmark (1000 images, 4 workers, batch_size=32):

Library	Throughput	Status
PyTorch DataLoader	48.07 img/s	✅ Baseline
TurboLoader v0.3.8	17.56 img/s	❌ 63% slower
TurboLoader TurboLoader (target)	150-200 img/s	🚧 In development

Identified Root Causes:

1. TAR mutex contention (75% impact) - all workers serialize on shared file handle
2. Memory allocation/copy overhead (20% impact)
3. Busy-wait spinning (10% impact)
4. Thread pool contention (10% impact)
5. JPEG decoder inefficiency (5% impact)

See GitHub Issue and ARCHITECTURE_V2.md for detailed analysis.

SIMD Transform Performance (Micro-benchmarks)

SIMD Transform Performance (Apple M1 Pro, NEON backend):

Operation	Throughput	Time per Image
Resize (256x256→224x224)	6,718 img/s	148.85 μs
Normalize (RGB)	47,438 img/s	21.08 μs

Test Configuration:

• Hardware: Apple M1 Pro (8 cores, 16GB RAM)
• Input: 256x256 RGB images
• Backend: NEON SIMD instructions
• All tests run on synthetic datasets

Note: These are micro-benchmarks of individual SIMD operations. End-to-end data loading performance is affected by the architectural issues described above.

---

Installation

pip install turboloader

Requirements:

• Python 3.8+
• C++20 compiler (GCC 10+, Clang 12+, MSVC 19.29+)
• CMake 3.15+

Optional Dependencies:

• libjpeg-turbo (JPEG decoding)
• libpng (PNG decoding)
• libwebp (WebP decoding)

---

Quick Start

Basic Usage

import turboloader

# Create pipeline
pipeline = turboloader.Pipeline(
    tar_paths=['imagenet.tar'],
    num_workers=8,
    batch_size=32,
    decode_jpeg=True
)

pipeline.start()

# Get batches
for _ in range(100):
    batch = pipeline.next_batch(32)
    for sample in batch:
        img = sample.get_image()  # NumPy array (H, W, C)
        # Your training code here...

pipeline.stop()

With SIMD Transforms

import turboloader

# Configure SIMD-accelerated transforms
config = turboloader.TransformConfig()
config.enable_resize = True
config.resize_width = 224
config.resize_height = 224
config.enable_normalize = True
config.mean = [0.485, 0.456, 0.406]
config.std = [0.229, 0.224, 0.225]

pipeline = turboloader.Pipeline(
    tar_paths=['imagenet.tar'],
    num_workers=8,
    decode_jpeg=True,
    enable_simd_transforms=True,
    transform_config=config
)

pipeline.start()
batch = pipeline.next_batch(256)
pipeline.stop()

With Data Augmentation

import turboloader

# Create augmentation pipeline
aug_pipeline = turboloader.AugmentationPipeline()
aug_pipeline.add_transform(turboloader.RandomHorizontalFlip(0.5))
aug_pipeline.add_transform(turboloader.ColorJitter(brightness=0.2, contrast=0.2))
aug_pipeline.add_transform(turboloader.RandomCrop(224, 224))

# Use with data loader (planned feature)
# pipeline = turboloader.Pipeline(tar_paths=['data.tar'], augmentations=aug_pipeline)

---

Architecture

TurboLoader is built on several high-performance components:

Core Components

1. Thread-Safe Concurrent Queues

   • Mutex-based synchronization for reliable multi-threaded operation
   • Thread-safe data passing between reader and worker threads
   • Stable performance with high worker counts (8+ workers)

2. Memory-Mapped I/O

   • mmap() for zero-copy file reading
   • Efficient TAR archive parsing
   • Minimizes memory allocations

3. SIMD Transforms

   • AVX2/AVX-512 on x86_64
   • NEON on ARM (Apple Silicon, ARM servers)
   • Vectorized resize, normalize, color conversion

4. Thread-Local Decoders

   • Per-thread JPEG/PNG/WebP decoders
   • Eliminates decoder allocation overhead
   • Maximizes cache locality

Supported Transforms

TurboLoader v0.3.x includes 7 SIMD-accelerated augmentation transforms:

• RandomHorizontalFlip: SIMD-optimized horizontal flip
• RandomVerticalFlip: SIMD-optimized vertical flip
• ColorJitter: Brightness, contrast, saturation adjustments
• RandomRotation: Bilinear interpolation rotation
• RandomCrop: Random crop with padding
• RandomErasing: Cutout augmentation
• GaussianBlur: Separable Gaussian filter (SIMD)

---

API Reference

Pipeline

class Pipeline:
    def __init__(
        self,
        tar_paths: List[str],
        num_workers: int = 4,
        queue_size: int = 256,
        shuffle: bool = False,
        decode_jpeg: bool = False,
        enable_simd_transforms: bool = False,
        transform_config: Optional[TransformConfig] = None
    )

    def start() -> None
    def stop() -> None
    def reset() -> None
    def next_batch(batch_size: int) -> List[Sample]
    def total_samples() -> int

TransformConfig

class TransformConfig:
    enable_resize: bool = False
    resize_width: int = 224
    resize_height: int = 224
    resize_method: ResizeMethod = ResizeMethod.BILINEAR

    enable_normalize: bool = False
    mean: List[float] = [0.0, 0.0, 0.0]
    std: List[float] = [1.0, 1.0, 1.0]

    enable_color_convert: bool = False
    src_color: ColorSpace = ColorSpace.RGB
    dst_color: ColorSpace = ColorSpace.RGB
    output_float: bool = False

Augmentation Transforms

class AugmentationPipeline:
    def __init__(seed: Optional[int] = None)
    def add_transform(transform: AugmentationTransform) -> None
    def clear() -> None
    def num_transforms() -> int

class RandomHorizontalFlip(AugmentationTransform):
    def __init__(probability: float = 0.5)

class ColorJitter(AugmentationTransform):
    def __init__(
        brightness: float = 0.0,
        contrast: float = 0.0,
        saturation: float = 0.0,
        hue: float = 0.0
    )

---

Roadmap

TurboLoader.0 (Q1 2025) - HIGH PRIORITY

Complete pipeline rewrite to fix critical performance issues

See ARCHITECTURE_V2.md for full design.

Core Infrastructure

• Lock-free SPSC ring buffers (~50x faster than mutex queues)
• Object pool for buffer reuse (eliminate malloc/free overhead)
• Zero-copy sample struct using std::span views

I/O Layer

• Per-worker TAR file handles (eliminate mutex bottleneck)
• Memory-mapped I/O for true zero-copy reads
• Worker-based sample partitioning

Decoding & Performance

• TurboJPEG SIMD decoder integration (2-3x faster)
• Object pool for decoded buffers
• Fallback to libjpeg for compatibility

Testing & Validation

• Comprehensive unit tests (all components)
• Performance benchmarks vs PyTorch (target: >100 img/s)
• Memory leak checks (valgrind)
• Thread safety verification (ThreadSanitizer)

Expected Performance: 150-200 img/s (3-4x faster than PyTorch baseline)

Estimated Timeline: 11-17 hours of development

Branch: TurboLoader-rewrite

---

v0.4.0 (Q2 2025)

• Full ImageNet benchmark suite
• TensorFlow/JAX integration
• Additional image formats (TIFF, BMP)
• Video decoding support

v0.5.0 (Q3 2025)

• GPU-accelerated JPEG decoding (nvJPEG)
• Distributed training support
• S3/GCS remote dataset loading

v1.0.0 (Q4 2025)

• Production-ready API stability
• Comprehensive documentation
• Full test coverage
• Performance optimization

---

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

Development Setup

# Clone repository
git clone https://github.com/ALJainProjects/TurboLoader.git
cd TurboLoader

# Install dependencies
brew install cmake libjpeg-turbo libpng libwebp  # macOS
# or
apt-get install cmake libjpeg-turbo8-dev libpng-dev libwebp-dev  # Ubuntu

# Build from source
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j8

# Run tests
./tests/turboloader_tests
./tests/test_simd_transforms

---

License

MIT License - see LICENSE for details.

---

Citation

If you use TurboLoader in your research, please cite:

@software{turboloader2025,
  author = {Jain, Arnav},
  title = {TurboLoader: High-Performance ML Data Loading},
  year = {2025},
  url = {https://github.com/ALJainProjects/TurboLoader}
}

---

Acknowledgments

• Inspired by FFCV and NVIDIA DALI
• Built with pybind11
• Uses libjpeg-turbo for fast JPEG decoding

---

Support

• Issues: GitHub Issues
• Documentation: docs/
• PyPI: https://pypi.org/project/turboloader/