Multi-agent cooperative games
Project description
CoGames: A Game Environment for the Alignment League Benchmark
The Alignment League Benchmark (ALB) is a suite of multi-agent games, designed to measure how well AI agents align, coordinate, and collaborate with others (both AIs and humans).
CoGames is the games environment for ALB. You can use it to:
- create new games
- train agents to play existing ALB games
- submit those agents to the ALB leaderboard
There's one ALB game right now: Cogs vs Clips.
Quick Start
Step 1: Install CoGames
Install cogames as a Python package.
pip install cogames
Using uv
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh
# Create a virtual environment
uv venv .venv
source .venv/bin/activate
# Install cogames
uv pip install cogames
Using Docker
# Ensure Python 3.12 is available
FROM python:3.12-slim
# Ensure C/C++ compiler is available
RUN apt-get update && \
apt-get install -y --no-install-recommends build-essential && \
rm -rf /var/lib/apt/lists/*
# Install cogames
RUN pip install --no-cache-dir cogames
Using Colab
Step 2: Play Cogs vs Clips
Play an easy mission in Cogs vs Clips using:
cogames tutorial play
The game will open in a new window, and the terminal will give you instructions to complete training mission.
Step 3: Submit a policy to the leaderboard
-
Log into the ALB leaderboard with your GitHub account.
cogames login -
Upload one of our starter policies under your name.
cogames upload --policy "class=cogames.policy.scripted_agent.starter_agent.StarterPolicy" --name "$USER.README-quickstart-starter-policy"
-
List the seasons currently running.
cogames seasons -
Submit the policy to one of the running seasons.
cogames submit "$USER.README-quickstart-starter-policy:v1" --season beta-cogsguard
Tutorials
To learn more, see:
- Creating a policy: Creating a custom policy and evaluating it
- Training: Training a custom policy and evaluating it
- Submitting: Submitting to the leaderboard and understanding the results
If you want help, or to share your experience, join the Discord.
About the game
CogsGuard is a cooperative territory-control game. Teams of AI agents ("Cogs") work together to capture and defend junctions against automated opponents ("Clips") by:
- gathering resources and depositing them at controlled junctions
- acquiring specialized roles (Miner, Aligner, Scrambler, Scout) at gear stations
- capturing neutral junctions using Aligners (costs 1 heart + 1 influence)
- disrupting enemy junctions using Scramblers (costs 1 heart)
- defending territory from Clips expansion
Read MISSION.md for a thorough description of the game mechanics.
There are many mission configurations available, with different map sizes, junction layouts, and game rules.
Overall, CogsGuard aims to present rich environments with:
- Territory control: Capture and defend junctions to score points each tick
- Role specialization: Four roles (Miner, Aligner, Scrambler, Scout) with distinct capabilities and dependencies
- Dense rewards: Agents receive reward every tick proportional to territory controlled
- Partial observability: Agents have limited visibility of the environment
- Required multi-agent cooperation: No single role can succeed alone; Miners need Aligners to capture junctions, Aligners need Miners for resources, Scramblers must clear enemy territory for Aligners to advance
About the tournament
How seasons work
The ALB leaderboard runs in seasons. Each season has two pools:
- Qualifying pool: Where new submissions start. Your policy plays matches against other policies in the pool.
- Competition pool: Policies that score above a threshold in qualifying get promoted here.
To see active seasons and their pools:
cogames seasons
How scoring works
When you submit a policy, it gets queued for matches against other policies in its pool. Our focal metric is VORP (Value Over Replacement Policy), which estimates how much your agent improves team performance compared to a baseline.
VORP is calculated by comparing:
- Replacement mean: The average score when only other pool policies play (no candidate)
- Candidate score: The score when your policy plays
The difference tells us how much value your policy adds to a team. A positive VORP means your policy makes teams better; a negative VORP means teams perform worse with your policy than without it.
You can evaluate VOR locally before submitting:
cogames pickup --policy <YOUR_POLICY> --pool <POOL_POLICY>
Viewing results
To check your submission status and match results:
cogames submissions
cogames leaderboard --season beta-cogsguard
Command Reference
Most commands are of the form cogames <command> --mission [MISSION] --policy [POLICY] [OPTIONS]
To specify a MISSION, you can:
- Use a mission name from the registry given by
cogames missions(e.g.training_facility_1). - Use a path to a mission configuration file (e.g.
path/to/mission.yaml). - Alternatively, specify a set of missions with
--mission-set.
To specify a POLICY, use one of two formats:
-
URI format (for checkpoint bundles):
- Point directly at a checkpoint bundle (directory or
.zipcontainingpolicy_spec.json) - Examples:
./train_dir/my_run:v5,./train_dir/my_run:v5.zip,s3://bucket/path/run:v5.zip - Use
:latestsuffix to auto-resolve the highest version:./train_dir/checkpoints:latest
- Point directly at a checkpoint bundle (directory or
-
Key-value format (for explicit class + weights):
class=: Policy shorthand or full class path fromcogames policies, e.g.class=lstmorclass=cogames.policy.random.RandomPolicy.data=: Optional path to a weights file (e.g.,weights.safetensors). Must be a file, not a directory.proportion=: Optional positive float specifying the relative share of agents that use this policy (default: 1.0).kw.<arg>=: Optional policy__init__keyword arguments (all values parsed as strings).
You can view all the commands with
cogames --help
and you can view help for a given command with:
cogames [COMMAND] --help
Missions Commands
cogames missions
Usage: cogames missions [OPTIONS] SITE
List available missions. This command has three modes: 1. List sites: Run with no arguments to see all available sites. 2. List missions at a site: Pass a site name (e.g., 'cogsguard_machina_1') to see its missions. 3. Describe a mission: Use -m to describe a specific mission. Only in this mode do --cogs, --variant, --format, and --save have any effect.
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ site TEXT Filter by site (e.g., cogsguard_machina_1) │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Describe ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission -m MISSION Mission to describe │ │ --cogs -c INTEGER Override agent count (requires -m) │ │ --variant -v VARIANT Apply variant (requires -m, repeatable) │ │ --difficulty LEVEL Difficulty (easy, medium, hard) controlling clips events (requires -m) │ │ --format [yaml|json] Output format (requires -m) │ │ --save -s PATH Save config to file (requires -m) │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames missions List all sites cogames missions cogsguard_machina_1 List missions at site cogames missions -m cogsguard_machina_1.basic Describe a mission cogames missions -m arena --format json Output as JSON
cogames evals
Usage: cogames evals [OPTIONS]
List all eval missions
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
cogames variants
Usage: cogames variants [OPTIONS]
List all available mission variants
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
cogames describe
Usage: cogames describe [OPTIONS] MISSION
Describe a mission and its configuration
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ * mission TEXT Mission name (e.g., hello_world.open_world) [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Configuration ─────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --cogs -c INTEGER Number of cogs (agents) │ │ --variant -v VARIANT Apply variant (repeatable) │ │ --difficulty LEVEL Difficulty (easy, medium, hard) controlling clips events │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames describe hello_world.open_world Describe mission cogames describe arena -c 4 -v dark_side With 4 cogs and variant
cogames make-mission
Usage: cogames make-mission [OPTIONS]
Create a custom mission from a base template
╭─ Mission ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission -m MISSION Base mission to start from │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Customization ─────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --cogs -c INTEGER RANGE [x>=1] Number of cogs (agents) │ │ --width INTEGER RANGE [x>=1] Map width │ │ --height INTEGER RANGE [x>=1] Map height │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --output -o PATH Output file path (.yml or .json) │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames make-mission -m hello_world -c 8 -o my_mission.yml 8 cogs cogames make-mission -m arena --width 64 --height 64 -o big.yml 64x64 map cogames play -m my_mission.yml Use custom mission
Play Commands
cogames play
Usage: cogames play [OPTIONS]
Play a game interactively. This runs a single episode of the game using the specified policy. By default, the policy is 'noop', so agents won't move unless manually controlled. To see agents move by themselves, use `--policy class=random` or `--policy class=baseline`. You can manually control the actions of a specific cog by clicking on a cog in GUI mode or pressing M in unicode mode and using your arrow or WASD keys. Log mode is non-interactive and doesn't support manual control.
╭─ Game Setup ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission -m MISSION Mission to play (run cogames missions to list) │ │ --variant -v VARIANT Apply variant modifier (repeatable) │ │ --difficulty LEVEL Difficulty (easy, medium, hard) controlling clips events │ │ --cogs -c N Number of cogs/agents [default: (from mission)] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Policy ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --policy -p POLICY Policy controlling cogs (noop, random, lstm, or path) [default: class=noop] │ │ --device DEVICE Policy device (auto, cpu, cuda, cuda:0, etc.) [default: auto] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Simulation ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --steps -s N Max steps per episode [default: 1000] │ │ --render -r [gui|vibescope|unicode|log|none] gui=MettaScope, vibescope=VibeScope, unicode=terminal, │ │ log=metrics only │ │ [default: gui] │ │ --seed INTEGER RNG seed for reproducibility [default: 42] │ │ --map-seed SEED Separate seed for procedural map generation │ │ [default: (same as --seed)] │ │ --autostart Start simulation immediately without waiting for user │ │ input │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --save-replay-dir DIR Save replay file for later viewing with cogames replay │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames play -m cogsguard_machina_1.basic Interactive cogames play -m cogsguard_machina_1.basic -p class=random Random policy cogames play -m cogsguard_machina_1.basic -c 4 -p class=baseline Baseline, 4 cogs cogames play -m cogsguard_machina_1 -r unicode Terminal mode
cogames replay
Usage: cogames replay [OPTIONS] FILE
Replay a saved game episode from a file in the GUI
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ * replay_path FILE Path to the replay file (.replay or .bin) [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames replay ./replays/game.replay Replay a saved game cogames replay ./train_dir/my_run/replay.bin Replay from training run
Evaluate Commands
cogames scrimmage
Usage: cogames scrimmage [OPTIONS]
Evaluate a single policy controlling all agents.
This command is equivalent to running `cogames run` with a single policy.
╭─ Mission ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission -m MISSION Missions to evaluate (supports wildcards) │ │ --mission-set -S SET Predefined set: integrated_evals, spanning_evals, diagnostic_evals, all │ │ --cogs -c N Number of cogs (agents) │ │ --variant -v VARIANT Mission variant (repeatable) │ │ --difficulty LEVEL Difficulty (easy, medium, hard) controlling clips events │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Policy ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --policy -p POLICY Policies to evaluate: (URI[,proportion=N] or NAME or │ │ class=NAME[,data=FILE][,proportion=N]...) │ │ --device DEVICE Policy device (auto, cpu, cuda, cuda:0, etc.) [default: auto] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Simulation ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --episodes -e N [x>=1] Number of evaluation episodes [default: 10] │ │ --steps -s N [x>=1] Max steps per episode [default: 1000] │ │ --seed N [x>=0] Seed for evaluation RNG [default: 42] │ │ --map-seed N [x>=0] MapGen seed for procedural maps [default: (same as --seed)] │ │ --action-timeout-ms MS [x>=1] Max ms per action before noop [default: 250] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --format FMT Output format: yaml or json │ │ --save-replay-dir DIR Directory to save replays │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames scrimmage -m arena.battle -p lstm Single policy eval
cogames run
Usage: cogames run [OPTIONS]
Evaluate one or more policies on missions. With multiple policies (e.g., 2 policies, 4 agents), each policy always controls 2 agents, but which agents swap between policies each episode. With one policy, this command is equivalent to `cogames scrimmage`.
╭─ Mission ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission -m MISSION Missions to evaluate (supports wildcards) │ │ --mission-set -S SET Predefined set: integrated_evals, spanning_evals, diagnostic_evals, all │ │ --cogs -c N Number of cogs (agents) │ │ --variant -v VARIANT Mission variant (repeatable) │ │ --difficulty LEVEL Difficulty (easy, medium, hard) controlling clips events │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Policy ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --policy -p POLICY Policies to evaluate: (URI[,proportion=N] or NAME or │ │ class=NAME[,data=FILE][,proportion=N]...) │ │ --device DEVICE Policy device (auto, cpu, cuda, cuda:0, etc.) [default: auto] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Simulation ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --episodes -e N [x>=1] Number of evaluation episodes [default: 10] │ │ --steps -s N [x>=1] Max steps per episode [default: 1000] │ │ --seed N [x>=0] Seed for evaluation RNG [default: 42] │ │ --map-seed N [x>=0] MapGen seed for procedural maps [default: (same as --seed)] │ │ --action-timeout-ms MS [x>=1] Max ms per action before noop [default: 250] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --format FMT Output format: yaml or json │ │ --save-replay-dir DIR Directory to save replays │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames run -m cogsguard_machina_1.basic -p lstm Evaluate single policy cogames run -m cogsguard_machina_1 -p ./train_dir/my_run:v5 Evaluate a checkpoint bundle cogames run -S integrated_evals -p ./train_dir/my_run:v5 Evaluate on mission set cogames run -m 'arena.*' -p lstm -p random -e 20 Evaluate multiple policies together cogames run -m cogsguard_machina_1 -p ./train_dir/my_run:v5,proportion=3 -p class=random,proportion=5 Evaluate policies in 3:5 mix
cogames pickup
Usage: cogames pickup [OPTIONS]
Evaluate a policy against a pool of other policies and compute VOR
╭─ Mission ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission -m MISSION Mission to evaluate on [default: cogsguard_machina_1.basic] │ │ --cogs -c N [x>=1] Number of cogs (agents) [default: 4] │ │ --variant -v VARIANT Mission variant (repeatable) │ │ --difficulty LEVEL Difficulty (easy, medium, hard) controlling clips events │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Policy ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --policy -p POLICY Candidate policy to evaluate │ │ --pool POLICY Pool policy (repeatable) │ │ --device DEVICE Policy device (auto, cpu, cuda, cuda:0, etc.) [default: auto] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Simulation ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --episodes -e N [x>=1] Episodes per scenario [default: 1] │ │ --steps -s N [x>=1] Max steps per episode [default: 1000] │ │ --seed N [x>=0] Base random seed [default: 50] │ │ --map-seed N [x>=0] MapGen seed for procedural maps [default: (same as --seed)] │ │ --action-timeout-ms MS [x>=1] Max ms per action before noop [default: 250] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --save-replay-dir DIR Directory to save replays │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames pickup -p greedy --pool random Test greedy against pool of random
cogames diagnose
Usage: cogames diagnose [OPTIONS] POLICY
Run diagnostic evals for a policy checkpoint
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ * policy TEXT Policy specification: URI (bundle dir or .zip) or NAME or │ │ class=NAME[,data=FILE][,kw.x=val] │ │ [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Evaluation ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission-set -S SET Eval suite to run [default: cogsguard_evals] │ │ --experiments NAME Specific experiments (subset of mission set) │ │ --cogs -c N Agent counts to test (repeatable) │ │ --device DEVICE Policy device (auto, cpu, cuda, cuda:0, etc.) [default: auto] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Simulation ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --steps -s N Max steps per episode [default: 1000] │ │ --episodes -e N Episodes per case [default: 3] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames diagnose ./train_dir/my_run Default CogsGuard evals cogames diagnose lstm -S diagnostic_evals Diagnostic evals (non-CogsGuard) cogames diagnose lstm -S tournament Tournament suite cogames diagnose lstm -c 4 -c 8 -e 5 Custom cog counts
Info Commands
cogames version
Usage: cogames version [OPTIONS]
Show version information for cogames and dependencies
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
cogames docs
Usage: cogames docs [OPTIONS] DOC
Print documentation (run without arguments to see available docs)
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ doc_name DOC Document name (readme, mission, technical_manual, scripted_agent, evals, mapgen) │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames docs List available documents cogames docs readme Print README cogames docs mission Print mission briefing
Policies Commands
cogames policies
Usage: cogames policies [OPTIONS]
Show available policy shorthand names
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Usage: Use these shorthand names with --policy or -p: cogames play -m arena -p class=random Use random policy cogames play -m arena -p class=baseline Use baseline policy
cogames validate-policy
Usage: cogames validate-policy [OPTIONS]
Validate the policy loads and runs for at least a single step
╭─ Policy ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ * --policy -p POLICY Policy specification: URI (bundle dir or .zip) or NAME or │ │ class=NAME[,data=FILE][,kw.x=val] │ │ [required] │ │ --device DEVICE Policy device (auto, cpu, cuda, cuda:0, etc.) [default: auto] │ │ --setup-script TEXT Path to a Python setup script to run before loading the policy │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Tournament ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --season SEASON Tournament season (determines which game to validate against) │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Server ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --server URL Tournament server URL (used to resolve default season) │ │ [default: https://api.observatory.softmax-research.net] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Tournament Commands
cogames login
Usage: cogames login [OPTIONS]
Authenticate with CoGames server
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --force -f Re-authenticate even if already logged in │ │ --timeout -t SECS Authentication timeout in seconds [default: 300] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Server ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --login-server URL Authentication server URL [default: https://softmax.com/api] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames login Authenticate with default server cogames login --force Re-authenticate even if already logged in
cogames submissions
Usage: cogames submissions [OPTIONS]
Show your uploads and tournament submissions
╭─ Filter ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --policy -p POLICY Filter by policy name (e.g., 'my-policy' or 'my-policy:v3') │ │ --season SEASON Filter by tournament season │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Server ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --login-server URL Authentication server URL [default: https://softmax.com/api] │ │ --server -s URL Tournament server URL [default: https://api.observatory.softmax-research.net] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --json Print raw JSON instead of table │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames submissions All your uploads cogames submissions --season beta-cogsguard Submissions in a season cogames submissions -p my-policy Info on a specific policy
cogames seasons
Usage: cogames seasons [OPTIONS] [SEASON_NAME]
List currently running tournament seasons
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ season_name [SEASON_NAME] Show versions of a specific season │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --versions -v List all versions of the season │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Server ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --login-server URL Authentication server URL [default: https://softmax.com/api] │ │ --server -s URL Tournament server URL [default: https://api.observatory.softmax-research.net] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --json Print raw JSON instead of table │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
cogames leaderboard
Usage: cogames leaderboard [OPTIONS]
Show tournament leaderboard for a season
╭─ Tournament ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --season SEASON Tournament season name [default: beta-cogsguard] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Server ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --login-server URL Authentication server URL [default: https://softmax.com/api] │ │ --server -s URL Tournament server URL [default: https://api.observatory.softmax-research.net] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --json Print raw JSON instead of table │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames leaderboard --season beta-cogsguard View rankings
cogames upload
Usage: cogames upload [OPTIONS]
Upload a policy to CoGames
╭─ Upload ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ * --name -n NAME Name for your uploaded policy [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Policy ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ * --policy -p POLICY Policy specification: URI (bundle dir or .zip) or NAME or │ │ class=NAME[,data=FILE][,kw.x=val] │ │ [required] │ │ --init-kwarg -k KEY=VAL Policy init kwargs (can be repeated) │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Files ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --include-files -f PATH Files or directories to include (can be repeated) │ │ --setup-script PATH Python setup script to run before loading the policy │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Tournament ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --season SEASON Tournament season (default: server's default season) │ │ --no-submit Upload without submitting to a season │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Validation ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --dry-run Run validation only without uploading │ │ --skip-validation Skip policy validation in isolated environment │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Server ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --login-server URL Authentication server URL [default: https://softmax.com/api] │ │ --server URL Tournament server URL [default: https://api.observatory.softmax-research.net] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames upload -p ./train_dir/my_run -n my-policy Upload and submit to default season cogames upload -p ./run -n my-policy --season beta-cvc Upload and submit to specific season cogames upload -p ./run -n my-policy --no-submit Upload without submitting cogames upload -p lstm -n my-lstm --dry-run Validate only
cogames submit
Usage: cogames submit [OPTIONS] POLICY
Submit a policy to a tournament season
╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ * policy_name POLICY Policy name (e.g., 'my-policy' or 'my-policy:v3' for specific version) [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Tournament ────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --season SEASON Tournament season name │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Server ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --login-server URL Authentication server URL [default: https://softmax.com/api] │ │ --server -s URL Tournament server URL [default: https://api.observatory.softmax-research.net] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames submit my-policy Submit to default season cogames submit my-policy:v3 --season beta-cvc Submit specific version to specific season
Tutorial Commands
cogames tutorial play
Usage: cogames tutorial play [OPTIONS]
Interactive tutorial - learn to play Cogs vs Clips
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
cogames tutorial cogsguard
Usage: cogames tutorial cogsguard [OPTIONS]
Interactive CogsGuard tutorial - learn roles and territory control
╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
cogames tutorial make-policy
Usage: cogames tutorial make-policy [OPTIONS]
Create a new policy from a template. Requires --trainable or --scripted.
╭─ Policy Type ───────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --trainable Create a trainable (neural network) policy │ │ --scripted Create a scripted (rule-based) policy │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --output -o FILE Output file path [default: my_policy.py] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames tutorial make-policy -t -o my_nn_policy.py Trainable (neural network) cogames tutorial make-policy -s -o my_scripted_policy.py Scripted (rule-based)
cogames tutorial train
Usage: cogames tutorial train [OPTIONS]
Train a policy on one or more missions. By default, our 'lstm' policy architecture is used. You can select a different architecture (like 'stateless' or 'baseline'), or define your own implementing the MultiAgentPolicy interface with a trainable network() method (see mettagrid/policy/policy.py). Continue training from a checkpoint using URI format, or load weights into an explicit class with class=...,data=... syntax. Supply repeated -m flags to create a training curriculum that rotates through missions. Use wildcards (*) in mission names to match multiple missions at once.
╭─ Mission Setup ─────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --mission -m MISSION Missions to train on (wildcards supported, repeatable for curriculum) │ │ --cogs -c N Number of cogs (agents) [default: (from mission)] │ │ --variant -v VARIANT Mission variant (repeatable) │ │ --difficulty LEVEL Difficulty (easy, medium, hard) controlling clips events │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Policy ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --policy -p POLICY Policy to train (URI (bundle dir or .zip) or NAME or │ │ class=NAME[,data=FILE][,kw.x=val]) │ │ [default: class=lstm] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Training ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --steps N [x>=1] Number of training steps [default: 10000000000] │ │ --batch-size N [x>=1] Batch size for training [default: 4096] │ │ --minibatch-size N [x>=1] Minibatch size for training [default: 4096] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Hardware ──────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --device DEVICE Device to train on (auto, cpu, cuda, mps) [default: cpu] │ │ --num-workers N [x>=1] Number of worker processes [default: (CPU cores)] │ │ --parallel-envs N [x>=1] Number of parallel environments │ │ --vector-batch-size N [x>=1] Vectorized environment batch size │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Reproducibility ───────────────────────────────────────────────────────────────────────────────────────────────╮ │ --seed N [x>=0] Seed for training RNG [default: 42] │ │ --map-seed N [x>=0] MapGen seed for procedural map layout [default: (same as --seed)] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Output ────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --checkpoints DIR Path to save training checkpoints [default: ./train_dir] │ │ --log-outputs Log training outputs │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Other ─────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --help -h Show this message and exit │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Examples: cogames tutorial train -m cogsguard_machina_1.basic Basic training cogames tutorial train -m cogsguard_machina_1.basic -p class=baseline Train baseline policy cogames tutorial train -p ./train_dir/my_run:v5 Continue from checkpoint cogames tutorial train -p class=lstm,data=./weights.safetensors Load weights into class cogames tutorial train -m mission_1 -m mission_2 Curriculum (rotates) Wildcard patterns: cogames tutorial train -m 'machina_2_bigger:*' All missions on machina_2_bigger cogames tutorial train -m '*:shaped' All "shaped" missions cogames tutorial train -m 'machina*:shaped' All "shaped" on machina maps
Citation
If you use CoGames in your research, please cite:
@software{cogames2025,
title={CoGames: Multi-Agent Cooperative Game Environments},
author={Softmax},
year={2025},
url={https://github.com/metta-ai/metta}
}
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
cogames-0.3.83.tar.gz
(11.0 MB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file cogames-0.3.83.tar.gz.
File metadata
- Download URL: cogames-0.3.83.tar.gz
- Upload date:
- Size: 11.0 MB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
861171a63d328f63f99bec1919507aa4b0102e958304c336118dfb23f537928d
|
|
| MD5 |
424d0a5ee6e94dd6bba6b8266b838e25
|
|
| BLAKE2b-256 |
e307852a8dfdb61e26406bcabd35c396731e009667f2e6c3af8643d3a0bd5ec9
|
Provenance
The following attestation bundles were made for cogames-0.3.83.tar.gz:
Publisher:
release-cogames.yml on Metta-AI/metta
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
cogames-0.3.83.tar.gz -
Subject digest:
861171a63d328f63f99bec1919507aa4b0102e958304c336118dfb23f537928d - Sigstore transparency entry: 939714769
- Sigstore integration time:
-
Permalink:
Metta-AI/metta@bda5bae1697c8f6589f3265f2ec3b1a18cddad8b -
Branch / Tag:
refs/tags/cogames-v0.3.83 - Owner: https://github.com/Metta-AI
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
self-hosted -
Publication workflow:
release-cogames.yml@bda5bae1697c8f6589f3265f2ec3b1a18cddad8b -
Trigger Event:
push
-
Statement type:
File details
Details for the file cogames-0.3.83-py3-none-any.whl.
File metadata
- Download URL: cogames-0.3.83-py3-none-any.whl
- Upload date:
- Size: 1.1 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
bf5f55131dc569a58fb30b6b14fb57e1146b0ce51bb54c4fea226bed204a88d5
|
|
| MD5 |
1685fb50ee34bf65c7ad44ebfedea767
|
|
| BLAKE2b-256 |
2d78fae914b2c90dc07904be3fbf5439d468f067b6f9f8cab8e47748fc1bf71a
|
Provenance
The following attestation bundles were made for cogames-0.3.83-py3-none-any.whl:
Publisher:
release-cogames.yml on Metta-AI/metta
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
cogames-0.3.83-py3-none-any.whl -
Subject digest:
bf5f55131dc569a58fb30b6b14fb57e1146b0ce51bb54c4fea226bed204a88d5 - Sigstore transparency entry: 939714782
- Sigstore integration time:
-
Permalink:
Metta-AI/metta@bda5bae1697c8f6589f3265f2ec3b1a18cddad8b -
Branch / Tag:
refs/tags/cogames-v0.3.83 - Owner: https://github.com/Metta-AI
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
self-hosted -
Publication workflow:
release-cogames.yml@bda5bae1697c8f6589f3265f2ec3b1a18cddad8b -
Trigger Event:
push
-
Statement type:
