nexuml 0.2.1

Composable ML pipelines for reproducible experiments.

NexuML is a modular PyTorch framework for building machine-learning systems from reusable, typed components connected through explicit TensorDict keys. A ScenarioSpec describes data, model pipelines, training, evaluation, logging, export, and execution in one place and can be persisted as validated YAML.

Documentation · Hands-on tutorials · TestPyPI

Why NexuML?

ML projects often start as a clean notebook and gradually collect project-specific dataset code, model wiring, training loops, evaluation scripts, and export logic. Copying a project template improves the folder structure, but it still duplicates implementations and makes experiments harder to reproduce or extend.

NexuML separates reusable implementations from experiment composition. Datasets, model blocks, evaluation algorithms, and loader backends can live in libraries; scenarios assemble those pieces into explicit pipelines without reimplementing the training lifecycle for every project.

Core ideas

• Typed component definitions — Python scenarios construct concrete Pydantic definitions directly; stable registry identities are used for discovery and persisted YAML.
• TensorDict pipelines — named tensors flow through ordered stages using explicit keys_in and keys_out contracts.
• Declarative scenarios — ScenarioSpec composes data, pipeline, training, evaluation, logging, checkpoint, export, and execution configuration.
• One Lightning lifecycle — PyTorch Lightning owns the training loop; local and Ray execution reuse the same NexuML session semantics.
• Pluggable data paths — PyTorch, NVIDIA DALI, and tensor-shard loaders plus dataset export to NumPy, mmap, Torch, TensorDict memmap, WebDataset, and tensor shards.
• Post-training evaluation — typed evaluation definitions materialize stateful algorithms while fitted pipeline layers can perform post-train processing before test.
• Portable model artifacts — export a compiled pipeline with weights, resolved configuration, metadata, and dependency information.
• CLI workflow — inspect registries and backends, resolve scenarios, build pipelines, train, tune, export datasets, and package models.

Install

NexuML is currently published on TestPyPI. PyPI remains the fallback for dependencies.

For most users, install the framework together with the reusable base library:

uv pip install --index https://test.pypi.org/simple --default-index https://pypi.org/simple "nexuml[library]"

Install only the framework and CLI when you want to provide all components yourself:

uv pip install --index https://test.pypi.org/simple --default-index https://pypi.org/simple nexuml

NVIDIA DALI, Ray, tracking, tuning, S3, and export integrations are optional. See the installation guide before adding platform-specific extras.

Container image

The complete Linux AMD64 development and training environment is also published with an explicit NexuML and CUDA version:

docker pull ghcr.io/nexufed/nexuml:0.2.0-cuda12.8.1
docker run --rm -it --gpus all ghcr.io/nexufed/nexuml:0.2.0-cuda12.8.1

The image contains both NexuML packages and all optional extras. See the installation guide for host requirements and other tags.

First look

With the base library installed, inspect a real scenario without starting a training job:

nexuml registry list scenarios
nexuml resolve cifar-resnet
nexuml build configs/cifar-resnet.yaml

This shows the central NexuML flow: a Python scenario is resolved to a reproducible configuration and then materialized into a validated TensorDict pipeline. Continue with Get started for training requirements.

Learn by building

The NexuML Tutorial repository is the home for complete hands-on projects. It builds an external NexuML library from scratch rather than hiding the framework behind finished built-in components.

The learning path starts with MNIST library basics, then adds file-backed Speech Commands audio with native DALI loading and demonstrates pipeline composition by swapping a CNN encoder for a Transformer while reusing the rest of the system.

Version note: the tutorial repository evolves independently from NexuML. NexuML 0.2 uses typed component definitions and rejects the legacy selector/parameter-bag syntax, so use a tutorial revision compatible with the NexuML version you install.

Documentation

• Get started — install NexuML and inspect your first scenario.
• Tutorials — complete hands-on projects and the tutorial compatibility note.
• Guides — accomplish a specific task such as training, tuning, exporting, or adding a component.
• Concepts — understand the architecture, TensorDict data flow, definitions, discovery, and scenarios.
• Reference — exact CLI, configuration, backend, decorator, and Python API information.

Extending NexuML

External libraries can provide their own typed layers, data sources, evaluation algorithms, loader backends, and scenarios through the nexuml.libraries entry-point group or a local library root. See Build a custom library for the package structure and component contracts.