laser-learning-environment 2.9.6

Laser Learning Environment (LLE) for Multi-Agent Reinforcement Learning

Laser Learning Environment (LLE)

Documentation: https://yamoling.github.io/lle/

LLE is a fast multi-agent reinforcement learning environment written in Rust. Agents start on start tiles, collect gems, and finish by reaching exit tiles. The available actions are North, South, East, West, and Stay.

When an agent enters a laser of its own colour, it blocks that beam. Otherwise, it dies and the game ends.

Quick start

Installation

Install the Laser Learning Environment with uv, pip, poetry, ...

pip install laser-learning-environment # Latest stable release with pip
pip install git+https://github.com/yamoling/lle # latest push on master

Usage

LLE can be used at two levels of abstraction: as an MARLEnv for cooperative multi-agent reinforcement learning or as a World for fine-grained control.

For cooperative multi-agent reinforcement learning

The LLE class wraps a World and implements the MARLEnv interface from the marlenv framework. Here is an example with the following map:

import lle

env = lle.from_str("S0 G X").build()
done = False
obs, state = env.reset()
while not done:
    # env.render() # Uncomment to render
    actions = env.sample_action()
    step = env.step(actions)
    # Access the step data with `step.obs`, `step.reward`, ...
    done = step.is_terminal # Either done or truncated

For other purposes or fine grained control

The World class provides fine grained control on the environment by exposing the state of the world and the events that happen when the agents move.

from lle import World, Action, EventType

world = World("S0 G X")  # Linear world with start S0, gem G and exit X
world.reset()
available_actions = world.available_actions()[0]  # [Action.STAY, Action.EAST]
events = world.step([Action.EAST])
assert events[0].event_type == EventType.GEM_COLLECTED
events = world.step([Action.EAST])
assert events[0].event_type == EventType.AGENT_EXIT

You can also access and force the state of the world:

state = world.get_state()
...
events = world.set_state(state)

You can query the world through properties such as world.start_pos, world.exit_pos, world.gems, world.lasers, and world.agents.

Generation and solving

The optional generator extra adds SAT-based generation and analysis helpers. The level6_style generator defaults to an exactly mutual cooperative configuration, and cooperation=True asks for any cooperative world:

pip install laser-learning-environment[generator]

import lle
from lle import CooperationLevel, World

world = lle.generate(kind="random", height=5, width=5, n_agents=2, seed=0)
plan = lle.solve(world, t_max=5)
assert plan is not None
assert lle.is_cooperative(World.level(6)) is True

coop = lle.generate(kind="random", height=6, width=6, n_agents=2, n_lasers=2, cooperation=True, seed=0)
assert lle.cooperation_level(coop) in CooperationLevel.cooperative_subtypes()

Citing our work

The environment has been presented at EWRL 2023 and at BNAIC 2023 where it received the best paper award.

@inproceedings{molinghen2023lle,
  title={Laser Learning Environment: A new environment for coordination-critical multi-agent tasks},
  author={Molinghen, Yannick and Avalos, Raphaël and Van Achter, Mark and Nowé, Ann and Lenaerts, Tom},
  year={2023},
  series={BeNeLux Artificial Intelligence Conference},
  booktitle={BNAIC 2023}
}

Development

If you want to modify the environment, you can clone the repo, install the python dependencies then compile it with maturin. The below example assumes that you are using uv as package manager but it should work with conda, poetry or just pip as well.

git clone https://github.com/yamoling/lle
uv venv         # create a virtual environment
source .venv/bin/activate
uv sync         # install python dependencies
maturin dev     # build and install lle in the venv

You can also re-generate the python bindings in the folder python/lle with

cargo run --bin stub-gen

Tests

Run unit tests in rust & python with

cargo test
maturin develop
pytest