MASA-Safe-RL

View Documentation

Welcome to MASA-Safe-RL, the Multi and Single Agent (MASA) Safe Reinforcement Learning library. The primary goal of this library is to develop a set of common constraints and environments for safe reinforcement learning research, built on top of the popular gymnasium interface. We span, CMPDs, probabilistic constraints, Reach-Avoid and LTL-Safety (DFA) properties.

The library is in very early stage development and we greatly appreciate and encourage feedback from the community about what they would like to see implemented. Currently we provide a set of basic tabular algroithms for safe RL, but we provide a modular and resuable framework for developing more complex algorithms and constraints.

If you use MASA-Safe-RL in your research please cite it in your publications.

@misc{Goodall2025MASASafeRL,
  title        = {{MASA-Safe-RL}: Multi and Single Agent Safe Reinforcement Learning},
  author       = {Goodall, Alexander W. and Adalat, Omar and Hamel De-le Court, Edwin and Belardinelli, Francesco},
  year         = {2025},
  howpublished = {\url{https://github.com/sacktock/MASA-Safe-RL/}},
  note         = {GitHub repository}
}

Quick Start

Installation

Prequisites

Python 3.13+ is required (3.13 recommended).

Installation with conda

• Install conda, e.g., via anaconda.
• Clone the repo:

git clone https://github.com/sacktock/MASA-Safe-RL.git
cd MASA-Safe-RL

• Create a conda virtual environment:

conda env create --name masa --file conda-environment.yaml
conda activate masa

• Install dependencies:

pip install -e .

Installation with uv

uv sync installs only base dependency package from the repository root.

Easiest way to install all dependencies: All dependency groups (docs and GPU support for CUDA13): uv sync --all-groups --no-group cuda12.

Alternatively, uv sync --all-groups --no-group cuda13 if your system only supports CUDA 12.

More minimal ways of installing dependency groups:

• For building docs, use uv sync --group docs from the repository root.
• Adding GPU support for Jax: uv sync --group cuda13 (or cuda12 if CUDA 13 is not supported by your device)

Installation with PyPI

MASA-Safe-RL is now available on PyPI: MASA-Safe-RL.

pip install -U MASA-Safe-RL

Enabling GPU Acceleration with JAX (Optional)

MASA-Safe-RL relies on JAX for GPU acceleration. If you are only interested in the gymnasium wrappers and constraints API then you do not need to complete the following steps.

Mac and Windows

macOS and Windows: we recommend JAX with CPU. No further action is required if you correctly followed the earlier steps.

pip

• Linux x86_64/aarch64 and Windows WSL2 x86_64: We strongly recommend using Ubuntu 22.04 or similar, jax and jaxlib should already be installed. You need to reinstall JAX based on your cuda driver compatibility. Do not use the -U option here as it may break earlier dependencies!

pip install "jax[cuda13]"

pip install "jax[cuda12]"

uv

Alternatively with uv,

uv sync --group cuda12

uv sync --group cuda13

How to run MASA

• You can run masa with the prebuilt CLI:

masa run --env-id bridge_crossing --algo ppo --seed 0

• You can run examples from the \examples folder via:

masa example prob_shield_example

Editing documentation

Documentation uses the root pyproject.toml and uv.lock, so you can build and serve it directly from the repository root.

uv run zensical build --strict

The built site will be available at site/index.html.

For live reload while editing, run:

uv run zensical serve

Local storage

The project currently uses the following directories for local storage:

• .cache/ - for persisting transition matrices to avoid recomputation
• videos/ - stores recorded videos from training runs at configured intervals or played runs using notebooks/
• runs/ - persisted data and metrics of training runs
• benchmarks/ - stores results from masa benchmark runs

Contributing

We use pre-commit for formatting and linting (Ruff, yamlfmt, taplo, mdformat, plus local checks). Install the git hook once per clone:

pip install pre-commit   # or: uv tool install pre-commit
pre-commit install

It then runs automatically on every git commit (run manually with pre-commit run --all-files). CI runs the same pinned hooks on changed files via pre-commit/action, alongside the test suite.

Getting in Touch

MASA-Safe-RL is primarliy managed by Alex Goodall and Omar Adalat. For correspondence in the early stages of the library we prefer you contact us directly via email (a.goodall22@imperial.ac.uk & o.adalat24@imperial.ac.uk), rather than raising issues on GitHub directly.

License

MASA-Safe-RL is released under Apache License 2.0.