tinyray

A minimal, actor-only Ray for ML experiments. Rust core, Python API.

import tinyray as tr

@tr.remote(num_gpus=1, max_restarts=3)
class Rollout:
    def __init__(self, cfg):
        self.env = make_env(cfg)
    def step(self):
        return self.env.rollout()          # ~10 MB

rollouts = tr.create_actors(Rollout, cfg, count=32)   # atomic, all or nothing
refs = [r.step.remote() for r in rollouts]            # returns immediately
ready, pending = tr.wait(refs, num_returns=24)        # 24 finished; 8 still running
learner.update.remote(ready)                          # data goes rollout -> learner

wait returns two lists of ObjectRef, never values: ready are the ones that have settled, pending the ones still running. Both are just names -- a task id and the address of the actor holding the result -- so the driver moves a few dozen bytes per reference and never the payload. Passing ready to the learner hands over those names, and the learner fetches each 10 MB batch straight from the rollout that produced it.

Dropping the eight slow rollouts drops their results, not their work: they are still running, and their outputs still occupy the producers' stores until the watermark or TTL reclaims them. And if the group runs NCCL collectives, those same eight actors must still attend the next barrier -- see tinyray.collective.

What it is, and is not

tinyray is an HTTP control plane. The data plane belongs to the framework.

SGLang, vLLM, Megatron and torchrun already do distributed compute well. What they want from a cluster manager is placement, rank assignment, supervision and restart -- and then to be left alone.

Actors only. No stateless tasks. ML workers hold models and CUDA contexts.
HTTP for control. At 200 ms per call a ~200 us round trip is 0.1% overhead, and every message is inspectable with ordinary tools.
It never claims a process-global resource. A process has one default torch.distributed group, one CUDA context, one set of signal handlers. Those belong to your framework. tinyray assigns ranks and injects the torchrun environment; you call init_process_group yourself.
It supervises processes it did not write. An SGLang server is an ordinary process: give it GPUs, wait until it actually serves, label its logs, restart it when it dies.
No object store. For pure-Python rollouts a result stays in the actor that produced it and consumers fetch it directly. When a framework owns the data, tinyray moves references and nothing else.

Minimal intrusion: your script keeps its own process

The least invasive option, and the one to reach for with Megatron, DeepSpeed or anything else that expects to own its entrypoint. The script is unchanged apart from one line:

# train.py -- your ordinary training script
import torch.distributed as dist

class Trainer:                                # not decorated, not subclassed
    def __init__(self):
        dist.init_process_group(backend="nccl")   # yours, not tinyray's
        self.model = build_model()                # yours
    def train_step(self, batch): ...

if __name__ == "__main__":
    import tinyray
    tinyray.serve(Trainer())                  # the only tinyray line

# controller.py
workers = tr.launch_workers(["python", "train.py"], size=8, gpus_per_worker=1)
workers.run("train_step", batch)              # all ranks, then awaited

server = tr.launch_process(                   # a process tinyray never imports
    ["python", "-m", "sglang.launch_server", "--port", "{port}", "--tp", "4"],
    name="rollout", num_gpus=4, ready_when="http:/health",
)

Nothing is pickled to the worker, no class is shipped over the wire, and __main__ stays yours. tinyray.connect(endpoint) will even drive a process that something else started.

examples/native_stack.py runs a four-rank trainer and an inference server end to end; it uses gloo and a toy server so it works without a GPU.

The actor API

Still available, and the right choice for code written for tinyray -- pure Python rollouts, evaluation harnesses, hyperparameter trials:

@tr.remote(num_gpus=1)
class Rollout:
    def step(self): return self.env.rollout()

rollouts = tr.create_actors(Rollout, count=32)

Here tinyray owns the process and constructs your class remotely, which is convenient and unavoidably more invasive.

Two details that are load-bearing rather than cosmetic:

Constructors run concurrently. A framework that rendezvous in __init__ blocks rank 0 until the last rank arrives, so constructing a group serially deadlocks on the first worker.
Readiness is observed, not assumed. An inference server binds its port minutes before it can answer; ready_when="http:/health" waits for the second event, not the first.

tinyray.collective still exists for pure-tinyray NCCL groups, but it takes the default process group and therefore cannot coexist with Megatron or SGLang. Use create_worker_group unless nothing else in the process wants that group.

Why Rust

An actor running a 200 ms training step holds the GIL. If the data path were Python, it could not serve result fetches during that time, and 32 actors fetching from each other would degrade to serial. The Rust core removes that coupling entirely:

10 MB decode	idle	4 GIL-bound threads	slowdown
native thread (the real serving path)	0.37 ms	0.38 ms	1.04x
initiated from Python	0.75 ms	36.7 ms	49x

Which also yields an architectural rule: the serving path must be driven by tokio, never by a Python-side loop. Work initiated from Python inherits GIL scheduling latency no matter how little of it runs in the interpreter, so /task/fetch never enters the interpreter at all.

Install

pip install tinyray

Wheels are built per interpreter version (3.9-3.13) for Linux and macOS on both x86_64 and aarch64. There is no abi3 build: the buffer protocol that makes results zero-copy is absent from the limited API on older interpreters, and version-specific wheels are a cheap price for it.

The package ships py.typed and hand-written stubs for the Rust extension, so type checkers see the full API.

From source

Requires a Rust toolchain.

python -m venv .venv
.venv/bin/pip install -e ".[dev]"
.venv/bin/maturin develop --release
pre-commit install

Tests

./scripts/test.sh                    # everything
cargo test --workspace               # 120 Rust tests
pytest tests/ -q                     # 216 Python tests
pytest benchmarks/ -q -s -m bench
python scripts/mutate.py             # 19 mutants: do the tests actually work?

Testing the tests

A green suite proves nothing about the tests. Three mechanisms keep them honest:

tests/test_suite_quality.py -- structural checks encoding the six blind spots that let real bugs through: unwired modules, timing constants that only ever run at their production value, options accepted but ignored, collapsed error taxonomies, untested lock branches, and design claims with no test.
tests/test_driver_byte_budget.py -- parks a 32 MB result in an actor, exercises every driver-side operation and asserts what crossed the driver's wire. The design's central claim is that payloads move between actors and never through the driver; this is where that claim is enforced for every operation rather than one.
scripts/mutate.py -- deliberately breaks 21 invariants and reports which ones the suite catches. It has already found three tests that passed while the behaviour they claimed to check was disabled.

Each check exists because a real bug got through in that exact shape.

Development

pre-commit install wires up the gates that CI also runs:

hook	what it guards
`ruff check` / `ruff format`	Python lint and formatting
`cargo fmt` / `cargo clippy -D warnings`	Rust formatting and lint
`mypy`	the public API's annotations, checked against the stubs

Releases are cut by pushing a v* tag; .github/workflows/release.yml runs the full suite, builds the wheel matrix and an sdist, and publishes to PyPI through a trusted publisher. workflow_dispatch targets TestPyPI by default so a dry run cannot reach the real index by accident.

Operations

tinyray status 127.0.0.1:41234 127.0.0.1:41235   # what is each actor doing?
tinyray introspect 127.0.0.1:41234               # raw report
tinyray health 127.0.0.1:41234

status calls out stalled callers, evictions, backpressure and stragglers, because "which actor is stuck, and on what?" is the question that dominates distributed ML debugging.

Layout

crates/tinyray-core/      wire protocol, framing, identifiers
crates/tinyray-runtime/   store, ordered queue, transport, cluster, collective, shm
crates/tinyray-py/        PyO3 bindings (all unsafe lives in buffers.rs)
python/tinyray/           API, serde, launcher, head, collective, pool, CLI

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tinyray-0.2.1.tar.gz (121.0 kB view details)

Uploaded Aug 14, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

tinyray-0.2.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.13manylinux: glibc 2.17+ x86-64

tinyray-0.2.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.13manylinux: glibc 2.17+ ARM64

tinyray-0.2.1-cp313-cp313-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (2.0 MB view details)

Uploaded Aug 14, 2026 CPython 3.13macOS 10.12+ universal2 (ARM64, x86-64)macOS 10.12+ x86-64macOS 11.0+ ARM64

tinyray-0.2.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.12manylinux: glibc 2.17+ x86-64

tinyray-0.2.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.12manylinux: glibc 2.17+ ARM64

tinyray-0.2.1-cp312-cp312-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (2.0 MB view details)

Uploaded Aug 14, 2026 CPython 3.12macOS 10.12+ universal2 (ARM64, x86-64)macOS 10.12+ x86-64macOS 11.0+ ARM64

tinyray-0.2.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.11manylinux: glibc 2.17+ x86-64

tinyray-0.2.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.11manylinux: glibc 2.17+ ARM64

tinyray-0.2.1-cp311-cp311-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (2.0 MB view details)

Uploaded Aug 14, 2026 CPython 3.11macOS 10.12+ universal2 (ARM64, x86-64)macOS 10.12+ x86-64macOS 11.0+ ARM64

tinyray-0.2.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.10manylinux: glibc 2.17+ x86-64

tinyray-0.2.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.10manylinux: glibc 2.17+ ARM64

tinyray-0.2.1-cp310-cp310-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (2.0 MB view details)

Uploaded Aug 14, 2026 CPython 3.10macOS 10.12+ universal2 (ARM64, x86-64)macOS 10.12+ x86-64macOS 11.0+ ARM64

tinyray-0.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.9manylinux: glibc 2.17+ x86-64

tinyray-0.2.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded Aug 14, 2026 CPython 3.9manylinux: glibc 2.17+ ARM64

tinyray-0.2.1-cp39-cp39-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (2.0 MB view details)

Uploaded Aug 14, 2026 CPython 3.9macOS 10.12+ universal2 (ARM64, x86-64)macOS 10.12+ x86-64macOS 11.0+ ARM64

File details

Details for the file tinyray-0.2.1.tar.gz.

File metadata

Download URL: tinyray-0.2.1.tar.gz
Upload date: Aug 14, 2026
Size: 121.0 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for tinyray-0.2.1.tar.gz
Algorithm	Hash digest
SHA256	`c5f18721b16ac20954b6f5b226d4a251ddc04912225c1c069ab5eb3bae47f384`
MD5	`ae49fc0c7c381d726594376a4266e707`
BLAKE2b-256	`e3f3ff75ee8d5c98e513e768e8189d888aa907b9ccb4e73ea7903e542338da5d`

Algorithm	Hash digest
SHA256	`4a8484f7822fdbc25fdb7b08dd2fca52f8a96bd43bce5683b45ca68e396ddb08`
MD5	`cf46e1e8a86c84d90d5504a000806bfd`
BLAKE2b-256	`d687b58a34b0d0051853eabae46971295ced3cf233762f7059b95f5d8bfa0489`

Algorithm	Hash digest
SHA256	`e653caaff69b3b744b9ef473e1c65578c026cb3bf8ba5c13f235525e323982d8`
MD5	`38f3d696346ae51ff5697c0043372971`
BLAKE2b-256	`b95c12efd6e290e3fb24bd0338deb22ac1c9f8f7d1c6bd425f07bb81693767b6`

Algorithm	Hash digest
SHA256	`2216705f59bf723b84838bf10f78c698cf47a3b571ba15415ce3534169c137c1`
MD5	`4b3ac5ba867fb6e51111b468baf2dcef`
BLAKE2b-256	`965aa4d426de23f4b7c06ff781c24168113a8f2cf05a75279d08d509d745c9e6`

Algorithm	Hash digest
SHA256	`28aaca97ea711b3347ae2b7b234e560f5b989e6538f4f08031c3e3cf2d934437`
MD5	`db6ef5fba4d6d51781617df3645c0f58`
BLAKE2b-256	`d283255fc2385d2238dca1ff5b8b83730753330f09bd621e6f978ba1cf77d6ed`

Algorithm	Hash digest
SHA256	`d54c0cf6ab383dc9422ca14bcc97feb1a4f1405feac49674c82c365ae29d5f58`
MD5	`e2def58061279d7bd4b4e43973789dd2`
BLAKE2b-256	`af263d184dccedd82db8c65ad8c8369f7bd172424515b342d70c8770d3a8259a`

Algorithm	Hash digest
SHA256	`3a8066de1ad7c82cabcc7653dca712c6f195b73188f7ebe53ce0b4a0aa505455`
MD5	`e5a58279eb47d1d12421df76629a7e3a`
BLAKE2b-256	`e88b409e5ed2d5a8719d95973a7628758b0ac656de041a1f41d52d0f0cc28727`

Algorithm	Hash digest
SHA256	`1decf6ba8b56d3cece46e3559f0ff085f0c4f9a27fcf0ea6cad16ba45cd8e490`
MD5	`378a12a9d27cdbd69201373def05e577`
BLAKE2b-256	`0ea7b65fb8609a4d7f09a5a62784e8c0f0a7351b686a1ed8a3c6b31e497fee88`

Algorithm	Hash digest
SHA256	`9dbc0d63bb9569714f88795e7c675eec8fc1f54f5ac29b00d4752920d6815393`
MD5	`227d28944c49f16916c75720d4aaf59a`
BLAKE2b-256	`aa0187cc2bada7affb0b24d933b588342512666a449cedf6d42a434461baf225`

Algorithm	Hash digest
SHA256	`5b21c6d537abb960b6f766b41014a75fe9b2068136829a24aa1e18168091e2fd`
MD5	`9b2ba0a331ce0926547b0e6f8dea43ec`
BLAKE2b-256	`23471669d31438b44668d93311014cb80045c0aa9729648dc163dc84c99cc695`

Algorithm	Hash digest
SHA256	`8f1d430444e273c1363ff98125e98a916e9718a665d77a2ccc8ce1adfb85550c`
MD5	`904cedd38bb88fdc6b306c9c04d16551`
BLAKE2b-256	`656e865d7821a1cbf342a7f8886b695f6cdbed8baa478c5cbc666bcfcc6d5ec4`

Algorithm	Hash digest
SHA256	`87c83b724ceba5a3524cf0ef507795474730c2d4e315e5ef67f9ede348d1b207`
MD5	`a1ce52ac88eb77fd192792aaf690c5a2`
BLAKE2b-256	`17dff9fc10b399c5c7d0b99be2030fd3e39ed63e0d53020a99a84107ae3159b2`

Algorithm	Hash digest
SHA256	`cf67918fe8286fd7a5058e5686bf5947d71fc09ec3c1599982b465c34d230b17`
MD5	`218218d1deed932641cbb82b34403149`
BLAKE2b-256	`caf581bb89112c904600bd882408e3c0e9bd7db605b5aae4dd793507eb2d91a2`

Algorithm	Hash digest
SHA256	`79681e3c571f4ecc96fb55d6fdda65390126850ad8f32723a06cfd2f8dfa4842`
MD5	`6e6c9628d4b30f503552a603c90a18a9`
BLAKE2b-256	`e1ef029835f74a2360debc400a16e887cb0cd9c6d3811b032b94bd952d1c75f9`

Algorithm	Hash digest
SHA256	`b65053e0cbf57c6c414668790dd1521ab5b2c77252afcce5acf9ed29b62f3a49`
MD5	`348cf8acf814e9f8c7b1877f0a0a0ae2`
BLAKE2b-256	`338bd7d257136607a8e7eda78700b6e480a9f2e5ec3814adf7b5e8400b2d7c06`

Algorithm	Hash digest
SHA256	`1662083c2c447bb21eaa78c2f3cac69bb48cf8ef6bb9ab10beefa2557f0de6dd`
MD5	`3f4e912a39c2754d82bd31c523218847`
BLAKE2b-256	`0b06e976287cc656270457f8a9c502987d6f534e19efd2179b14e1f10bf0a077`

tinyray 0.2.1

tinyray

What it is, and is not

Minimal intrusion: your script keeps its own process

The actor API

Why Rust

Install

From source

Tests

Testing the tests

Development

Operations

Layout

Project links

Key dates

1 maintainer

GitHub Statistics

License

Requires

Provides Extra

Tags

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes