FootsiesGym

A reinforcement learning environment for HiFight's Footsies game. This environment serves as a benchmark for multi-agent reinforcement learning in a two-player zero-sum fighting game. For a full description, see: FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games.

The environment wraps the open-source Unity implementation, augmented with a gRPC server controlled through a Python harness. Training is implemented using Ray's RLlib.

Installation

uv add footsies-gym          # or: pip install footsies-gym

Or install from source:

git clone https://github.com/como-research/FootsiesGym.git
cd FootsiesGym
uv sync                      # or: pip install -e .

Game binaries are downloaded automatically on first use and verified with SHA256 checksums — no manual binary setup is required on Linux or macOS.

Quick Start

import footsiesgym
from footsiesgym.footsies.game import constants

# Create environment (downloads binaries automatically)
env = footsiesgym.make(platform="linux")

obs, infos = env.reset()

while True:
    actions = {agent: env.action_space[agent].sample() for agent in env.agents}
    obs, rewards, terminateds, truncateds, infos = env.step(actions)

    if terminateds["__all__"] or truncateds["__all__"]:
        obs, infos = env.reset()

Note: launch_binaries=True (the default) works on Linux and macOS (on macOS the server runs under Rosetta and is re-signed automatically; see Platform Support). Windows is not supported.

Configuration

Creating an Environment

Use footsiesgym.make() for a quick setup with sensible defaults:

env = footsiesgym.make(
    config={...},           # Override default config keys (see below)
    platform="linux",       # "linux" or "mac"
    launch_binaries=True,   # Auto-launch game server (Linux and macOS)
)

Or create the environment directly for full control:

from footsiesgym import FootsiesEnv

env = FootsiesEnv(config={...})

Config Options

Action Space

Each agent selects from a Discrete action space:

The action space is Discrete(6) by default, or Discrete(9) with use_special_charge_action=True.

Special Charge Mechanic

When use_special_charge_action=True, agents can hold the attack button to charge a special attack (requires 60 frames / 15 steps at frame_skip=4). SPECIAL_CHARGE is a toggle: activating it holds the attack input, and all movement actions become their attack variants (e.g., FORWARD becomes FORWARD_ATTACK). Toggle again to release.

Action Delay

Actions are queued and executed after action_delay // frame_skip steps. This simulates reaction time and makes the environment more realistic.

Observation Space

Each agent receives a Box observation of shape (88,) containing:

Observations are asymmetric: each agent sees its own privileged information but not the opponent's.

Rewards

Rewards are zero-sum between the two agents (rewards["p1"] + rewards["p2"] == 0).

When use_reward_budget=True, guard break rewards are deducted from the win reward so total reward per episode is capped at win_reward_scaling_coeff. When False, guard break rewards are additive.

Platform Support

Manual Launch

Binaries are downloaded, extracted, re-signed (ad-hoc), and launched automatically, just like on Linux — the game server runs under Rosetta since gRPC (Grpc.Core) is x86_64-only:

env = footsiesgym.make(platform="mac")

To launch the game server manually instead, download the binaries from the CDN or the binaries-v1 GitHub release:

curl -LO https://footsiesgym.chasemcd.com/v0.7.0/footsies_mac_headless_bbdb506.zip
# or: gh release download binaries-v1 --repo como-research/FootsiesGym --pattern "footsies_mac_*"

unzip footsies_mac_headless_bbdb506.zip
arch -x86_64 footsies_mac_headless_bbdb506/FOOTSIES --port 50051 -batchmode --grpc

If macOS reports "This will damage your computer," re-sign the binary:

codesign --force --deep --sign - footsies_mac_headless_bbdb506/FOOTSIES

Then create the environment against the running server:

env = footsiesgym.make(
    config={"port": 50051, "headless": True},
    platform="mac",
    launch_binaries=False,
)

Binary Management

All binaries are hosted on a CDN (footsiesgym.chasemcd.com), with GitHub Releases as a fallback source, and are downloaded automatically on first use. Downloads are verified with SHA256 checksums.

Offline usage: The binaries must be downloaded at least once before running offline. The easiest way to ensure this is to run the environment once while online so the binaries are automatically downloaded and cached.

Training

[!NOTE] The code for the experiments from the corresponding paper is coming soon; it was not run through the RLlib or CleanRL examples provided.

Training uses Ray RLlib with the APPO algorithm.

Training

RLlib

Two RLlib training stacks are available: the newer RLModule-based stack and the legacy ModelV2-based stack.

# RLModule stack (recommended)
python -m experimentation.experiments.rllib.train_rlmodule --experiment-name <experiment-name>

# Legacy ModelV2 stack
python -m experimentation.experiments.rllib.train --experiment-name <experiment-name>

# Local debug mode (single env runner)
python -m experimentation.experiments.rllib.train_rlmodule --experiment-name <experiment-name> --debug

CleanRL

A self-contained CleanRL PPO example is also included.

System Architecture

Solid arrows are the per-step data flow; dotted arrows run once, on first use. The environment steps the game frame_skip frames per call, and in vectorized mode (num_envs > 1) a single server hosts N games stepped in one batched RPC.

Throughput

Linux	macOS

Environment steps per second vs. number of parallel environments, with $P$ concurrent game-server processes. See benchmarking/ to reproduce these results. When increasing $P$, we launch additional game servers. For example, $P=2$ launches the Footsies game binary twice. The number of parallel environments corresponds to the num_envs environment configuration paramer.

Development

The project is managed with uv:

uv sync --all-extras         # create the venv with all extras
uv run pytest                # run the test suite (-m "not slow" to skip server tests)
uv build                     # build sdist + wheel
uv publish                   # publish to PyPI

Citation

If you use FootsiesGym in your research, please cite:

@article{mcdonald2026footsies,
  title={FootsiesGym: A Fighting Game Benchmark for Two-Player
               Zero-Sum Imperfect-Information Games},
  author    = {McDonald, Chase and Tsang, Nathan and Kerr, Wesley N.},
  journal={arXiv preprint arXiv:2607.06514},
  year={2026}
}

License

This project is licensed under the GNU General Public License v3.0.

FootsiesGym is based on the open-source Footsies game by HiFight.