Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

REAX

Coverage Tests Documentation Latest Version https://img.shields.io/pypi/pyversions/reax.svg https://img.shields.io/pypi/l/reax.svg https://img.shields.io/badge/code%20style-black-000000.svg

REAX — Scalable, flexible training for JAX, inspired by the simplicity of PyTorch Lightning.

REAX - Scalable Training for JAX

REAX is a minimal and high-performance framework for training JAX models, designed to simplify research workflows. Inspired by PyTorch Lightning, it brings similar high-level abstractions and scalability to JAX users, making it easier to scale models across multiple GPUs with minimal boilerplate. 🚀

A Port of PyTorch Lightning to JAX

Much of REAX is built by porting the best practices and abstractions of PyTorch Lightning to the JAX ecosystem. If you’re familiar with PyTorch Lightning, you’ll recognize concepts like:

  • Training loops ⚡

  • Multi-GPU training 🖥️

  • Logging and checkpointing 💾

However, REAX has been designed with JAX-specific optimizations, ensuring high performance without sacrificing flexibility.

Why REAX? 🌟

  • Scalable: Built to leverage JAX’s parallelism and scalability. ⚡

  • Minimal Boilerplate: Simplifies the training process with just enough structure. 🧩

  • Familiar: For users who have experience with frameworks like PyTorch Lightning, the transition to REAX is seamless. 🔄

Installation 🛠️

To install REAX, run the following command:

pip install reax

REAX example

Define the training workflow. Here’s a toy example:

# main.py
from functools import partial
import jax, optax, reax, flax.linen as linen
from reax.demos import mnist


class Autoencoder(linen.Module):
    def setup(self):
        super().__init__()
        self.encoder = linen.Sequential([linen.Dense(128), linen.relu, linen.Dense(3)])
        self.decoder = linen.Sequential([linen.Dense(128), linen.relu, linen.Dense(28 * 28)])

    def __call__(self, x):
        z = self.encoder(x)
        return self.decoder(z)


# --------------------------------
# Step 1: Define a REAX Module
# --------------------------------
# A ReaxModule (nn.Module subclass) defines a full *system*
# (ie: an LLM, diffusion model, autoencoder, or simple image classifier).
class ReaxAutoEncoder(reax.Module):
    def __init__(self):
        super().__init__()
        self.ae = Autoencoder()

    def configure_model(self, stage: reax.Stage, batch, /):
        if self.parameters() is None:
            # Prepare example batch for initialization
            x, _ = batch
            x = x.reshape(x.shape[0], -1)
            # Initialize Flax Linen model with RNGs and example input
            params = self.ae.init(self.rngs(), x)
            self.set_parameters(params)

    def __call__(self, x):
        # Use the full autoencoder model for forward pass
        return self.ae.apply(self.parameters(), x)

    def training_step(self, batch, batch_idx):
        x, _ = batch
        x = x.reshape(x.shape[0], -1)
        # Static method receives params, data, and model apply function
        loss, grads = jax.value_and_grad(self.loss_fn, argnums=0)(
            self.parameters(), x, self.ae.apply
        )
        self.log("train_loss", loss, on_step=True, prog_bar=True)
        return loss, grads

    @staticmethod
    @partial(jax.jit, static_argnums=2)
    def loss_fn(params, x, apply_fn):
        """Compute reconstruction loss.

        Static method for JIT compilation. Receives parameters and apply function.
        """
        predictions = apply_fn(params, x)
        return optax.losses.squared_error(predictions, x).mean()

    def configure_optimizers(self):
        opt = optax.adam(learning_rate=1e-3)
        state = opt.init(self.parameters())
        return opt, state


# -------------------
# Step 2: Define data
# -------------------
dataset = mnist.MnistDataset(download=True)
trainer = reax.Trainer()
train, val = reax.data.random_split(trainer.rngs, dataset, [55000, 5000])

# -------------------
# Step 3: Train
# -------------------
autoencoder = ReaxAutoEncoder()
trainer.fit(autoencoder, reax.ReaxDataLoader(train), reax.ReaxDataLoader(val))

Here, we reproduce an example from PyTorch Lightning, so we use torch vision to fetch the data, but for real models there’s no need to use this or pytorch at all.

Disclaimer ⚠️

REAX takes inspiration from PyTorch Lightning, and large portions of its core functionality are directly ported from Lightning. If you are already familiar with Lightning, you’ll feel right at home with REAX, but we’ve tailored it to work seamlessly with JAX’s performance optimizations.

Metadata

Release files for reax 0.7.0.dev1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for reax 0.7.0.dev1
File Size Uploaded Provenance Actions
reax-0.7.0.dev1.tar.gz 140.5 kB Without provenance Details for reax-0.7.0.dev1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for reax 0.7.0.dev1
File Interpreter ABI Platform Actions
reax-0.7.0.dev1-py3-none-any.whl Python 3 none any Details for reax-0.7.0.dev1-py3-none-any.whl

Total release size: 322.9 kB

Release files / reax-0.7.0.dev1.tar.gz

Download URL reax-0.7.0.dev1.tar.gz
Size 140.5 kB
Tags Source
SHA-256 checksum
How to use checksums
06a2590eee6c9919acc221e18af36913f1689f61fdda4a9eb33d0e1ec26dd82e
BLAKE2b-256 checksum
How to use checksums
baed110bfbe6cc2966743842e2a835b5e56560b03c6907b406f029201290dd46
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.14

Release files / reax-0.7.0.dev1-py3-none-any.whl

Download URL reax-0.7.0.dev1-py3-none-any.whl
Size 182.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c1fb8e057d694691fdb67346c3bb8ac241f0b1131551a5e72782dee63aa9bbf4
BLAKE2b-256 checksum
How to use checksums
8ad64de73844bf5ff291abc789ffbb1d74e0c5810a0409713c16d54a6067fd1b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.14

Security for reax 0.7.0.dev1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page