laser-learning-environment 0.2.5

A fast Multi-Agent Reinforcement Learning Environment

Laser Learning Environment (LLE)

LLE is a fast Multi-Agent Reinforcement Learning environment written in Rust which has proven to be a difficult exploration benchmark so far. The agents start in the start tiles, must collect the gems and finish the game by reaching the exit tiles. There are five actions: North, South, East, West and Stay.

When an agent enters a laser of its own colour, it blocks it. Otherwise, it dies and the game ends.

Quick start

Installation

You can install the Laser Learning Environment with pip or poetry.

pip install laser-learning-environment # Stable release with pip
poetry add laser-learning-environment  # Stable release with poetry
pip install git+https://github.com/yamoling/lle # latest push on master

Usage

LLE can be used at two levels of abstraction: as an RLEnv for cooperative multi-agent reinforcement learning or as a World for many other purposes.

For cooperative multi-agent reinforcement learning

The LLE class inherits from the RLEnv class in the rlenv framework. Here is an example with the following map:

from lle import LLE

env = LLE.from_str("S0 G X")
done = truncated = False
obs = env.reset()
while not (done or truncated):
    # env.render() # Uncomment to render
    actions = env.action_space.sample(env.available_actions())
    obs, reward, done, truncated, info = env.step(actions)

For other purposes or fine grained control

The World class provides fine grained control on the environment by exposing the state of the world and the events that happen when the agents move.

from lle import World, Action, EventType

world = World("S0 G X")  # Linear world with start S0, gem G and exit X
world.reset()
available_actions = world.available_actions()[0]  # [Action.STAY, Action.EAST]
events = world.step([Action.EAST])
assert events[0].event_type == EventType.GEM_COLLECTED
events = world.step([Action.EAST])
assert events[0].event_type == EventType.AGENT_EXIT

You can also access and force the state of the world

state = world.get_state()
...
events = world.set_state(state)

You can query the world on the tiles with world.start_pos, world.exit_pos, world.gem_pos, ...

Citing our work

The environment has been presented at EWRL 2023 and at BNAIC 2023 where it received the best paper award.

@inproceedings{molinghen2023lle,
  title={Laser Learning Environment: A new environment for coordination-critical multi-agent tasks},
  author={Molinghen, Yannick and Avalos, Raphaël and Van Achter, Mark and Nowé, Ann and Lenaerts, Tom},
  year={2023},
  series={BeNeLux Artificial Intelligence Conference},
  booktitle={BNAIC 2023}
}

Development

If you want to modify the environment, you can clone the repo, install the python dependencies then compile it with maturin.

git clone https://github.com/yamoling/lle
poetry shell # start the virtual environment
poetry install
maturin develop # install lle locally

Building

This project has been set up using Poetry. To build the project, run the following commands:

poetry shell
poetry install
maturin develop  # For development
maturin build    # For distribution

Tests

This project does not respect Rust unit tests convention and takes inspiration from this structure. Unit tests are in the src/unit_tests folder and are explicitely linked to in each file with the #path directive. Integration tests are written on the python side.

Run unit tests with

cargo test

Run integration tests with

maturin develop
pytest