Skip to main content

RAMJET — Distributed peer cache for ML training data

Python PyPI License

RAMJET is a peer-to-peer cache between any ML training job and any S3-compatible object store. The first run pulls data from S3 once and seeds the cluster; subsequent runs across the team serve every sample peer-to-peer with zero S3 calls.

Works with any framework that loads data from S3/HTTP/local paths — PyTorch, TensorFlow, JAX, HuggingFace Datasets, Ultralytics, custom loaders — and any DDP launcher (torchrun, DeepSpeed, Accelerate, SLURM).

Measured on a real 2× A5000 cluster (5315 samples, 5 epochs): 6.5× faster export when cache is warm, zero S3 requests after the first run.

Why RAMJET?

Problem Solution
Repeated S3 pulls across team experiments One pull seeds the cluster; the rest serve peer-to-peer
Network bottleneck from shared object storage Local SSD cache on every training node
No visibility into data-loading bottlenecks Live dashboard with per-node bytes-by-source split (local / peer / S3) and Grafana-native Prometheus metrics
Multi-node DDP coordination Auto-detect rank/world size from torchrun/SLURM env

Quick Start

1. Install

pip install ramjetio

2. Add to Your Training Script

import ramjetio
from torch.utils.data import DataLoader

ramjetio.init()

dataset = ramjetio.CachedDataset(your_dataset)
loader = DataLoader(dataset, batch_size=32)

for batch in loader:
    train_step(batch)

3. Run

Get your API key from app.ramjet.io (create a cluster → copy key).

export RAMJET_API_KEY="your_api_key_here"
python train.py

Multi-GPU: torchrun --nproc_per_node=N train.py

That's it! Your nodes will appear in the dashboard within seconds.

How It Works

   ┌──────────┐   ┌──────────┐   ┌──────────┐
   │  Node 0  │   │  Node 1  │   │  Node 2  │
   │  train   │   │  train   │   │  train   │
   │    │     │   │    │     │   │    │     │
   │    ▼     │   │    ▼     │   │    ▼     │
   │ ramjetio │◄─►│ ramjetio │◄─►│ ramjetio │
   │  cache   │   │  cache   │   │  cache   │
   │ NVMe SSD │   │ NVMe SSD │   │ NVMe SSD │
   └─────┬────┘   └─────┬────┘   └─────┬────┘
         └──────────────┼──────────────┘
                        ▼
                ┌───────────────┐
                │ S3 / MinIO /  │
                │ R2 / GCS / …  │
                └───────────────┘

   Hits stay local or hop to a peer (sub-ms over LAN).
   Only the first miss in the cluster ever touches object storage.

Features

  • 🚀 Zero-config caching — ramjetio.init() handles everything
  • 📊 Real-time dashboard — monitor cache hits, throughput, GPU utilization
  • 🔄 Consistent hashing — data distributed evenly across nodes
  • 💾 Disk-backed cache — survives restarts, uses NVMe SSDs efficiently
  • 🔌 Works with any setup — torchrun, DeepSpeed, Accelerate, custom launchers
  • ☁️ S3/MinIO integration — configure data source in dashboard, not in code

Formats — what counts as a training example

A CachedDataset needs to know how your bucket turns into a list of training examples. That's a Format. Format owns the storage layout (which keys make up one example and how to fetch their bytes); your training code owns the task (decode, parse labels into targets, model, loss).

Use the built-in or write your own:

  • GenericFormat — each object under the data-source prefix is its own example ({'data': bytes}). For standalone files. Does not pair files.
ds = ramjetio.CachedDataset(format=ramjetio.GenericFormat(), dataset_name="blobs")

If an example is split across several objects (an image plus a sidecar label, say), write your own Format subclass — point list_examples at your keys. See docs/formats.md for the full contract and a worked custom example.

Integration Examples

Runnable scripts in examples/:

See docs/INTEGRATION.md for deeper walkthroughs.

Configuration

Environment Variables

Variable Description Default
RAMJET_API_KEY Your API key (required) —
RAMJET_CACHE_PATH Local cache directory /tmp/ramjet_cache
RAMJET_CACHE_SIZE Max cache size 100GB
RAMJET_PORT Cache server port 9000
RAMJET_STORE Local store backend: diskcache (default) or lmdb (opt-in, ~15× faster GET p50 on warm reads; see docs/STORE.md) diskcache
OTEL_EXPORTER_OTLP_ENDPOINT OTLP collector endpoint. When set (and ramjetio[otel] is installed), enables distributed tracing across SDK → peer → backend. See docs/TRACING.md. (unset, tracing disabled)
RAMJET_TRACE_SAMPLE Trace sampling ratio (0.0–1.0). Only meaningful when tracing is enabled. 1.0

Dashboard Settings

Configure in the web dashboard (no code changes needed):

  • Data Source: S3/MinIO endpoint, bucket, credentials
  • Cache Settings: TTL, replication factor, eviction policy

Distributed Training (DDP)

RAMJET automatically detects torchrun and DDP environments:

Single Machine, Multiple GPUs (torchrun)

# 4 GPUs on one machine
torchrun --nproc_per_node=4 train.py
import ramjetio
import torch.distributed as dist

# Only LOCAL_RANK=0 starts cache server - others wait and share it
ramjetio.init()

# All ranks use the same cache
dataset = ramjetio.CachedDataset(your_dataset)

Multi-Node Training

RAMJET auto-detects your cluster manager — no manual configuration needed:

Environment How to launch RAMJET detects it?
SLURM srun python train.py ✅ Automatic
Kubernetes (PyTorchJob) Managed by operator ✅ Automatic
DeepSpeed deepspeed --hostfile hosts train.py ✅ Automatic
Accelerate accelerate launch train.py ✅ Automatic
torchrun torchrun --nproc_per_node=N train.py ✅ Automatic
SageMaker Configured in SageMaker console ✅ Automatic

Each node runs one cache server (on LOCAL_RANK=0), and all nodes share data via consistent hashing. RAMJET reads LOCAL_RANK, RANK, WORLD_SIZE from environment — every major launcher sets these automatically.

CLI Tools

# Start cache server manually (usually not needed — ramjetio.init() does this)
ramjetio-server --port 9000 --capacity 100GB

# Check cache status
ramjetio-client stats

# Clear cache
ramjetio-client clear

Requirements

  • Python 3.8+
  • PyTorch 1.9+
  • Linux (recommended for production)
  • SSD storage for cache (recommended)

Documentation

License

PolyForm Noncommercial License 1.0.0 — free for personal and non-commercial use. For commercial licensing, contact licensing@ramjet.dev. See LICENSE for details.

Support

Metadata

Release files for ramjetio 0.13.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ramjetio 0.13.0
File Size Uploaded Provenance Actions
ramjetio-0.13.0.tar.gz 147.2 kB Without provenance Details for ramjetio-0.13.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ramjetio 0.13.0
File Interpreter ABI Platform Actions
ramjetio-0.13.0-py3-none-any.whl Python 3 none any Details for ramjetio-0.13.0-py3-none-any.whl

Total release size: 260.3 kB

Release files / ramjetio-0.13.0.tar.gz

Download URL ramjetio-0.13.0.tar.gz
Size 147.2 kB
Tags Source
SHA-256 checksum
How to use checksums
4ad2207f5fa042049c774f8423ece2a3e0d1d0875244c718d20a6a7d49a13579
BLAKE2b-256 checksum
How to use checksums
7da58eb1113e91d7dc51f835cddb2d6a5c4c6e950bb529f35cdeb0d80e2ae13b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.11

Release files / ramjetio-0.13.0-py3-none-any.whl

Download URL ramjetio-0.13.0-py3-none-any.whl
Size 113.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
4757e5ce5cf91621c1a8f89671c18790511259f41002b499becc3da93ff008ac
BLAKE2b-256 checksum
How to use checksums
38b3c7a4a12a96115cd7aab7c211b2c487b61501ad28a41e9daf4a999a2458ce
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.12.11

Security for ramjetio 0.13.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page