marlenv - A unified framework for multi-agent reinforcement learning
Documentation: https://yamoling.github.io/multi-agent-rlenv
marlenv is a strongly typed library for multi-agent and multi-objective reinforcement learning.
Install the library with:
$ pip install multi-agent-rlenv # Basics
$ pip install multi-agent-rlenv[all] # With all optional dependencies
$ pip install multi-agent-rlenv[smac,overcooked] # Only SMAC & Overcooked
It aims to provide a simple and consistent interface for reinforcement learning environments by providing abstraction models such as Observations or Episodes. marlenv provides adapters for popular libraries such as gym or pettingzoo and provides utility wrappers to add functionalities such as video recording or limiting the number of steps.
Most classes are dataclasses, which makes serialization straightforward (for example with orjson).
Fundamentals
States & Observations
MARLEnv.reset() returns a pair of (Observation, State) and MARLEnv.step() returns a Step.
Observationcontains:data: shape[n_agents, *observation_shape]available_actions: boolean mask[n_agents, n_actions]extras: extra features per agent (default shape(n_agents, 0))
Staterepresents the environment state and can also carryextras.Stepbundlesobs,state,reward,done,truncated, andinfo.
Rewards are stored as np.float32 arrays. Multi-objective envs use reward vectors with reward_space.size > 1.
Extras
Extras are auxiliary features appended by wrappers (agent id, last action, time ratio, available actions, ...).
Wrappers that add extras must update both extras_shape and extras_meanings so downstream users can interpret them.
State extras should stay in sync with Observation extras when applicable.
Environment catalog
marlenv.catalog exposes curated environments and lazily imports optional dependencies.
from marlenv import catalog
env1 = catalog.overcooked().from_layout("scenario4")
env2 = catalog.lle().level(6)
env3 = catalog.DeepSea(max_depth=5)
env4 = catalog.connect_n()(width=7, height=6, n=4)
Catalog entries require their corresponding extras at install time (e.g., multi-agent-rlenv[overcooked], multi-agent-rlenv[lle]).
Wrappers & builders
Wrappers are composable through RLEnvWrapper and can be chained via Builder for fluent configuration.
from marlenv import Builder
from marlenv.adapters import SMAC
env = (
Builder(SMAC("3m"))
.agent_id()
.time_limit(20)
.available_actions()
.build()
)
Common wrappers include time limits, delayed rewards, masking available actions, and video recording.
Parallel environments
ParallelMARLEnv steps several environments with the same inputs and outputs, and batches their observations so that a policy can select the actions of all environments in a single forward pass.
from marlenv import Builder, ParallelMARLEnv
from marlenv.catalog import DeepSea, EnvPool
env = ParallelMARLEnv.from_factory(lambda: Builder(DeepSea(max_depth=5)).time_limit(20).build(), n_envs=8)
# Or, from an existing pool: env = EnvPool([...]).to_parallel()
obs, state = env.reset(seed=0) # obs.data has shape [n_envs, n_agents, *observation_shape]
while True:
step = env.step(policy(obs)) # One action per environment: [n_envs, ...]
if step.is_terminal: # All environments are done or truncated
break
obs = step.obs
Environments are not reset automatically: once an environment is done or truncated, it is no longer stepped and its actions are ignored until the next reset().
Each ParallelStep has per-environment done and truncated flags, a mask of the environments that were actually stepped, and the individual Step of each environment in steps (None for masked environments).
Using the library
Adapters for existing libraries
Adapters normalize external APIs into MARLEnv:
import marlenv
gym_env = marlenv.make("CartPole-v1", seed=25)
from marlenv.adapters import SMAC
smac_env = SMAC("3m", debug=True, difficulty="9")
from pettingzoo.sisl import pursuit_v4
from marlenv.adapters import PettingZoo
env = PettingZoo(pursuit_v4.parallel_env())
For deterministic behavior, seed the environment:
env.seed(123)
obs, state = env.reset()
Designing a custom environment
Create a custom environment by inheriting from MARLEnv and implementing reset, step, get_observation, and get_state.
import numpy as np
from marlenv import MARLEnv, DiscreteSpace, MultiDiscreteSpace, Observation, State, Step
class CustomEnv(MARLEnv[MultiDiscreteSpace]):
def __init__(self):
super().__init__(
n_agents=3,
action_space=DiscreteSpace.action(5).repeat(3),
observation_shape=(4,),
state_shape=(2,),
)
self.t = 0
def reset(self, * seed:int|None=None):
if seed is not None:
self.seed(seed)
self.t = 0
return self.get_observation(), self.get_state()
def step(self, action):
self.t += 1
return Step(self.get_observation(), self.get_state(), reward=0.0, done=False)
def get_observation(self):
return Observation(np.zeros((3, 4), dtype=np.float32), self.available_actions())
def get_state(self):
return State(np.array([self.t, 0], dtype=np.float32))
Related projects
- MARL: Collection of multi-agent reinforcement learning algorithms based on
marlenvhttps://github.com/yamoling/marl - Laser Learning Environment: a multi-agent gridworld that leverages
marlenv's capabilities https://pypi.org/project/laser-learning-environment/
1 maintainer
Metadata
1 maintainer
Release files for multi-agent-rlenv 4.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| multi_agent_rlenv-4.4.0.tar.gz | 63.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| multi_agent_rlenv-4.4.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 131.7 kB
Release files / multi_agent_rlenv-4.4.0.tar.gz
| Download URL | multi_agent_rlenv-4.4.0.tar.gz |
|---|---|
| Size | 63.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4ae16d652f5ab6528488a6de8bd81f5ce3af18093fb4fd91c59aebc6dc069390
|
|
BLAKE2b-256 checksum How to use checksums |
eebda070761e9e2e06a07f1478698734a127b0e6feba810269cb621de52ebd9b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 2, 2026.
Transparency logRelease files / multi_agent_rlenv-4.4.0-py3-none-any.whl
| Download URL | multi_agent_rlenv-4.4.0-py3-none-any.whl |
|---|---|
| Size | 68.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
68422f009281fe8ddde958eaf8fdf7fb16b661ea1f53eeeabef1926d3a98d831
|
|
BLAKE2b-256 checksum How to use checksums |
1608e18ce8c030a058a70e56d956f142fbd894bceb4bf13e6d00cee74a9c4e5c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 2, 2026.
Transparency log
