TrackmaniaRL

TrackmaniaRL is a reinforcement-learning library for training agents in Trackmania 2020. It combines ready-to-use algorithms, replay buffers, model families and Trackmania telemetry with explicit interfaces for replacing any component in an experiment.

The current release is available on PyPI. TrackmaniaRL requires Python 3.12 or newer.

What you get

• asynchronous local or distributed actor/learner training;
• SAC, REDQ-SAC, TQC, IQN and stable discrete SAC learners;
• uniform, prioritized, sequence and demonstration-mixing replay;
• typed configuration, transitions and training batches;
• Trackmania telemetry, lidar and track-geometry feature pipelines;
• durable rollout journals, safe policy transfer and resumable checkpoints;
• local JSONL observability with optional W&B, Captum, Gemini and Optuna integrations;
• an installable extension project generated by trackmaniarl init.

TrackmaniaRL has no global runtime configuration and no mandatory external tracker. A run is described by run.yaml and explicit module:attribute component paths.

Install and create an agent

Install the published CLI with uv:

uv tool install trackmaniarl
trackmaniarl init my-trackmania-agent --template trackmania
cd my-trackmania-agent
uv sync
uv run trackmaniarl validate run.yaml

The trackmania template creates a commented, installable agent project with the Trackmania, algorithm, distributed and W&B extras declared for you. Omit --template trackmania to generate the smaller, game-free starter project. trackmaniarl validate checks imports, contracts and a synthetic learner update without starting the game or contacting an external tracker.

To add the SDK to an existing Python project instead, choose only the extras you need:

uv add trackmaniarl
uv add "trackmaniarl[algorithms,distributed]"

Run Trackmania

Live collection requires Trackmania 2020 on Windows, the bundled OpenPlanet plugin and a prepared map/geometry asset. Follow the Trackmania workflow or the concrete agent OpenPlanet guide before starting the game integration.

The generated Trackmania project pins the patched Palamabron/vgamepad revision containing the unreleased Windows installation fix from vgamepad PR #47. Keep that source pin until the fix is included in an upstream vgamepad release.

With Trackmania and the OpenPlanet plugin running:

uv run trackmaniarl track check
uv run trackmaniarl smoke run.yaml --transitions 100
uv run trackmaniarl train run.yaml

The bounded smoke test uses the same asynchronous learner/actor path as training, verifies a live policy refresh and writes a checkpoint. Start a fresh run directory when the run API or immutable configuration changes; the current schema is RunSpec 1.2.

On Windows, a generated project selects the tested CUDA PyTorch wheels. Linux uses CPU wheels by default and can host an offline or remote learner. ROCm users must select the matching AMD Torch index; macOS uses the normal PyPI wheel and can use MPS. device: auto resolves CUDA, ROCm, MPS or CPU from the installed Torch build.

Runtime model

run.yaml -> coordinator/learner -> SQLite WAL -> replay -> update -> checkpoint
              ^       |
              |       +---- safetensors policy snapshot
              |
              +---- local or remote actors -> durable rollout spool

trackmaniarl train starts a coordinator/learner and one local actor as independent, Windows-safe spawn processes. Collection continues while the learner updates replay and periodically publishes policy snapshots.

For multiple machines, set the same TRACKMANIARL_DISTRIBUTED_TOKEN on every participant and expose the learner through an encrypted tunnel. The learner binds to loopback so its bearer token and rollout data are not sent over the network in clear text:

# training machine
uv run trackmaniarl learner run.yaml --bind 127.0.0.1:8787

# Trackmania machine: create the tunnel first
ssh -N -L 8787:127.0.0.1:8787 TRAINING_MACHINE
uv run trackmaniarl actor run.yaml --connect 127.0.0.1:8787 --actor-id PC-1

The handshake rejects mismatched run fingerprints, map UIDs, geometry hashes and feature/action contracts. Rollouts use Protobuf/gRPC with Zstandard compression, and policy state is transferred with safetensors rather than pickle.

Components and extension API

trackmaniarl.builtins is the supported catalogue of bundled algorithms, models, feature pipelines and replay strategies. A component can also be referenced directly, for example:

components:
  learner:
    class_path: trackmaniarl.algorithms.implicit_quantile_q_learning:ImplicitQuantileQLearning

The stable contracts in trackmaniarl.core include Learner, Policy, ModelFactory, ReplayStore, Sampler, FeaturePipeline, Evaluator, RunLogger and CheckpointCodec. Game-specific implementations belong in the generated extension project, so offline validation does not require Trackmania or optional game dependencies.

Every run writes a redacted immutable manifest, local JSONL events, checkpoints and bounded compressed episode artifacts. Only the learner needs W&B credentials; WANDB_API_KEY can be supplied through the environment or project .env.

See the SDK guide for the full component schema and a built-in run example. Release history is in the changelog.

Development

Clone the repository and install the development group:

git clone https://github.com/Palamabron/AITrackmania.git
cd AITrackmania
uv sync --group dev
uv run poe fmt
uv run poe types
uv run poe test

The commands are intentionally identical on Windows, Linux, WSL and CI. See CONTRIBUTING.md and SECURITY.md before opening a contribution or reporting a vulnerability.

Project status and attribution

TrackmaniaRL is beta software. The project originated from TMRL and has since been substantially redesigned. It is not affiliated with or endorsed by Ubisoft, Nadeo or the TMRL maintainers. Trackmania is a trademark of Nadeo/Ubisoft. See NOTICE for attribution.