MARL-BattleGrounds

The JAX-native Benchmark for Heterogeneous and Competitive Multi-agent Reinforcement Learning.

Installation | Quick Start | Baselines | Guides | See Also | Citation

MARL-BattleGrounds (MARL-BGs) is an easy-to-use, JAX-native benchmark for heterogeneous and competitive team-vs-team multi-agent reinforcement learning (MARL). It is designed to be sample-efficient, computationally efficient and researcher-centric. Most MARL benchmarks are either fast but simple, like SMAX, where winning comes down to unit micromanagement, or rich but costly, like Dota 2, which took OpenAI Five ten months of training on large clusters. MARL-BGs fills the missing middle: rich team play, with fast and cheap experiments on one GPU.

• The Game. In Team Deathmatch, two teams of up to five agents fight on 52 hand-designed maps. Each agent is a Mage, Warrior, Hunter, Rogue or Priest. Each class has its own Basic ability, passive and Ultimate, built to help allies and counter enemies, so a team wins by working together. Obstacles block sight and attacks, so each agent sees only part of the map. Maps 0-11 form a curriculum, maps 0-41 are for training, maps 42-46 for validation and maps 47-51 for testing.
• The Evaluation. Eight hand-crafted endgame scenarios, each with a verified winning line, test skills such as agent modeling and multi-step planning. A monthly tournament retrains every entry on equal compute, plays it against the current "Big Nine" on the test maps, and rates it with draw-aware Elo and confidence intervals. Every entrant is released as a downloadable opponent. Up to 11,192 metrics per game, in 28 topics, show how a team won, such as how much the Priest healed each ally.
• The Tools. Bring your own learner through plain JAX reset and step, or start from six tuned baselines. Their trained Season 0 models, and every save from 0 to 250 million steps, load by name from Hugging Face. Training helpers cover self-play, opponent pools, curricula and reward shaping. One call evaluates any mix of policies, so cross-play and zero-shot coordination tests come out of the box. MARL-BGs also ships three strong scripted opponents, an LLM API that turns a language model into a team, the BattleClient for playing alongside or against your policies, and the analytical Replay Viewer for stepping through any game tick by tick.

Try it in your browser: the Colab notebook walks through the API, the speed, training a team and the trained Season 0 teams.

Installation

Start here:

You do not need to install JAX first: each route installs the right JAX for your machine.

On Linux, an Apple Silicon Mac or Windows with WSL2, one command installs everything and opens the BattleClient. It picks the right JAX build for your GPU:

curl -L https://github.com/arm-2-lab/MARL-BattleGrounds/archive/refs/tags/v1.0.1.tar.gz | tar xz
cd MARL-BattleGrounds-1.0.1 && sh start.sh

In your own Python project (Python 3.12 to 3.14; 3.14 recommended):

pip install "marl-battlegrounds[all]"           # CPU
pip install "marl-battlegrounds[all,cuda13]"    # NVIDIA GPU, driver 580 or newer

With Docker or Podman, on any system (add --gpus all and the 1.0.1-cuda12 tag for an NVIDIA GPU):

docker run --rm -it -p 127.0.0.1:8765:8766 -p 127.0.0.1:8767:8768 -v "${PWD}:/work" ghcr.io/arm-2-lab/marl-battlegrounds:1.0.1

"Tested" means we ran exactly these instructions on that system for this release.

Docker runs everything: play, replays, training, evaluation, tournaments, analysis and notebooks. Its limits:

Every route, the GPU builds, Windows, clusters and offline use: the install guide.

Quick Start

make, reset and step use JaxMARL's call order (reset(key), step(key, state, actions)), and compile under jit, vmap and lax.scan.

import jax
import marl_battlegrounds as marl_bgs

env = marl_bgs.make("tdm", map_id=0, num_envs=4)  # four games at once
key = jax.random.key(0)
observation, state = env.reset(key)
for _ in range(50):
    key, act_key, step_key, reset_key = jax.random.split(key, 4)
    actions = env.sample_actions(act_key, state)  # legal random actions
    observation, state, reward, done, info = env.step(step_key, state, actions)
    observation, state = env.reset_done(reset_key, state)  # restart finished games

On one RTX 5090 with 1,024 parallel 5v5 games, the simulator alone runs 50,416 game steps per second (random legal actions, compilation excluded), and recurrent MAPPO trains at 29,878 game steps per second in pure self-play.

Train

One command trains recurrent IPPO with the Season 0 learner settings (250 million game steps) against itself, its past copies and the three scripted opponents, and keeps its best checkpoint on the validation maps. What Training Costs gives the time and memory on one GPU.

python -m marl_battlegrounds.experiments.train method=ippo pool=scripted \
  'validation_opponents=[tdm-alpha,tdm-beta,tdm-gamma]' output_dir=runs/my-ippo

Evaluate

import marl_battlegrounds as marl_bgs

results = marl_bgs.evaluate("random", "tdm-alpha", num_episodes=32, num_envs=32)
print(results.head_to_head())  # games, wins, draws, losses, point margin

Your method is always Team A, and paired games swap the spawn ends, so a map's layout cannot favor either side.

Watch And Play

python -m marl_battlegrounds replay path/to/game.marlbg-replay.json

import marl_battlegrounds as marl_bgs

marl_bgs.battle_client(team_b="tdm-alpha")  # play in your browser against a scripted team

Baselines

Every save of the six Season 0 models, from 0 to 250 million steps, loads by name from Hugging Face, for example marl_bgs.load_method("MARL-BattleGrounds/tdm-season-0-recurrent-ippo@final") (Released Models).

Your own method can be anything that turns permitted observations into legal actions: a network, a planner, a language model or a mix.

Guides

Reproduce The Paper

The paper's commands, frozen settings and result tables are in scripts/paper_experiments/.

See Also

JAX-native environments:

• JaxMARL: Multi-Agent RL Environments and Algorithms in JAX.
• Gymnax: classic RL environments in JAX.
• Jumanji: combinatorial and planning environments in JAX.
• Pgx: board games in JAX.
• Brax: physics simulation for robot learning in JAX.
• XLand-MiniGrid: open-ended grid worlds for meta-RL in JAX.
• Craftax: an open-ended survival game in JAX.

JAX-native algorithms:

• Mava: JAX implementations of popular MARL algorithms.
• PureJaxRL: JAX implementation of PPO, and demonstration of end-to-end JAX-based RL training.

Citation

If you use MARL-BattleGrounds in your work, please cite the paper:

@article{hawili2026marlbattlegrounds,
  title   = {{MARL-BattleGrounds}: A Novel {JAX}-native Benchmark for Heterogeneous and Competitive Multi-agent {RL}},
  author  = {Hawili, Ulixes Tariq and Ramamoorthy, Subramanian and Altmann, Yoann and Carlucho, Ignacio},
  journal = {arXiv preprint},
  month   = oct,
  year    = {2026}
}

Contribute

Pull requests are welcome. Start with CONTRIBUTING.md, and open an issue for bugs, ideas, questions and leaderboard submissions.

License

Licensed under the Apache License, Version 2.0.