gradlab 0.2.2

🤖 RL workbench for training game agents 🎮

GradLab is a Python CLI and reproducible reinforcement-learning workbench for researchers who train, evaluate, compare, inspect, and publish game agents. It turns versioned goal contracts and recipes into traceable local or queued runs, with portable policies and evidence-backed results.

Try the bundled ROM-free smoke recipe without cloning, credentials, or a ROM:

uvx gradlab@0.2.2 train gradlab__bandit/ppo

The run writes a directly playable policy below ~/.config/gradlab/runs/.

Install

Install uv, then run:

git clone https://github.com/tsilva/gradlab.git
cd gradlab
./install.sh
gradlab validate

Run gradlab --help to open the command reference, or train and play the bundled smoke recipe:

gradlab train gradlab__bandit/ppo
gradlab play --recipe gradlab__bandit/ppo

gradlab play starts the local web player and prints its loopback URL.

Gymnasium discrete environments

GradLab includes strict Turbo-vector goals for ten Gymnasium discrete-action environments:

gradlab train CartPole-v1/ppo
gradlab train MountainCar-v0/ppo
gradlab train Acrobot-v1/ppo
gradlab train LunarLander-v3/ppo
gradlab train FrozenLake-v1/ppo
gradlab train FrozenLake8x8-v1/ppo
gradlab train CliffWalking-v1/ppo
gradlab train CliffWalkingSlippery-v1/ppo
gradlab train Taxi-v3/ppo
gradlab train Blackjack-v1/ppo

Use gymnasium:<environment-id> for the qualified environment ID. These goals use isolated spawned lanes, explicit masked resets, native rewards, and RGB rendering through the same training, checkpoint-publication, and playback workflows as other GradLab providers. They can establish Training Success but cannot establish Acceptance, Promotion, or Release. Native categorical observations remain categorical; Blackjack uses the fixed MultiDiscrete([32, 11, 2]) encoding (player sum, dealer card, usable ace).

Commands

gradlab train <goal>/<recipe>       # train a checked-in recipe locally
gradlab play [artifact]             # browse or inspect local and remote policies
gradlab validate                    # validate goals, recipes, benchmarks, and ops config
gradlab env list                    # list available environment providers
gradlab rom status --json           # inspect registered ROM assets
gradlab benchmark list              # list reproducible benchmark profiles
gradlab experiment status --run ID  # inspect an orchestrated run
uv run pytest                       # run Python tests
uv run ruff check .                 # lint Python code
pnpm test:web                       # run web-player tests

Use gradlab <command> --help for full arguments. Gameplay datasets, leader queries, W&B reports, and workspace management are also available through the dataset, leaders, reports, and workspaces commands.

Research results

Start with Featured Research on Hugging Face or its YouTube playlist.

Environment indexes:

• VizdoomDeathmatch-v1 models and videos
• SuperMarioBros-Nes-v0 models and videos

Evaluation evidence and representative replay are distinct: immutable model tags contain the accepted evaluation record, while videos show one separately labeled episode.

Queued runs

Queued training uses dstack for placement, a single supervisor-controlled training container, Modal for separately scheduled checkpoint evaluation, R2 for run authority and artifacts, and W&B for metrics.

Copy the portable operator template into private user configuration and run the read-only preflight before launching:

mkdir -p ~/.config/gradlab
install -m 600 ops/operator.example.toml ~/.config/gradlab/operator.toml
gradlab experiment operator-preflight --json

Then launch a checked-in recipe using its resolved goal and bounded defaults:

gradlab experiment launch \
  --recipe-file experiments/goals/SuperMarioBros-Nes-v0/Level1-1/recipes/ppo.yaml \
  --follow

The recipe path identifies its owning goal. The launch seed defaults to 12, the description defaults to <goal> <recipe> seed <seed>, compute defaults to the configured local fleet, and maximum duration defaults to 48 hours. --follow keeps the command attached after submission, streams authoritative run-state changes, and exits when training and its terminal drain are complete. Interrupting the client stops the stream without canceling the remote run. Omit the flag to retain asynchronous submission. Add --json for typed JSON Lines events (launch, status, and terminal).

When the selected coordinator declares ssh_tunnel metadata in private operator.toml, this same command reuses an existing local endpoint or opens a temporary SSH tunnel automatically. The tunnel remains attached for the full command and is closed on exit only when GradLab created it.

Local compute requires an enrolled fleet in ~/.config/gradlab/instances.md. Paid cloud compute is always bounded and explicitly authorized. See COMPUTE.md and the dstack runbook before operating queued runs.

Notes

• GradLab requires Python 3.14 and uses uv with a committed lockfile and a seven-day dependency age gate. Supported binary targets are macOS arm64 and Linux x86_64.
• Local gradlab train Runs disable W&B and acceptance evaluation by default. They can establish Training Success but cannot establish Acceptance or Promotion.
• gradlab.ppo is the opt-in tensor-native PPO backend. It accepts the sb3.ppo configuration surface plus precision (fp32, amp-fp16, or amp-bf16) and an execution_profile. sb3-parity preserves SB3's eager, unfused, environment-major minibatch path; compiled-parity and compiled-fused-parity isolate the CUDA optimizations; max-throughput additionally uses GPU-native permutation and is the default. The backend keeps PPO artifacts mutually resumable with SB3 and uses eager execution on CPU or MPS. Checked-in training recipes remain on sb3.ppo until the dedicated RTX 4090 throughput gate passes.
• NES recipes require a lawfully obtained ROM supplied with --rom-path or registered with gradlab rom sync. ROMs and credentials must remain outside source control.
• Generated runs default to ~/.config/gradlab/runs/; other generated logs and models belong in ignored logs/ and models/ directories.
• dstack task success is not scientific success. A queued run succeeds only when its terminal receipt proves checkpoint publication, evaluation drain, promotion state, and metric delivery.
• SPECS.md defines project-wide product requirements, docs/specs/ defines scoped product requirements, METRICS.md defines metric semantics, and experiments/README.md explains the checked-in research contracts.

Architecture

License

GradLab is licensed under the MIT License. Third-party attributions are listed in THIRD_PARTY_NOTICES.md.