Welcome to the future of urban routing.

Multi-agent reinforcement learning framework for modeling and simulating the collective route choices of humans and autonomous vehicles.

RouteRL is a multi-agent reinforcement learning framework that integrates RL-based collective route choice with a microscopic traffic simulation, SUMO, facilitating the testing and development of efficient route choice strategies. The proposed framework simulates the daily route choices of driver agents in a city, including two types:

• human drivers, emulated using discrete choice models,
• and AVs, modeled as MARL agents optimizing their policies for a predefined objective.

RouteRL aims to advance research in MARL, traffic assignment problems, social reinforcement learning, and human-AI interaction for transportation applications.

• The main class is TrafficEnvironment and is a PettingZoo AEC API environment.
• There are two types of agents in the environment and are both represented by the BaseAgent class.
  • Human drivers are simulated using human route-choice behavior from transportation research.
  • Automated vehicles (AVs) are the RL agents that aim to optimize their routes and learn the most efficient paths.
• RouteRL is compatible with popular RL libraries such as TorchRL.

For overview see the paper and for more details, check the documentation online.

RouteRL usage and functionalities at a glance

The following is a simplified code of a possible standard MARL algorithm implementation via TorchRL.

env = TrafficEnvironment(seed=42, **env_params) # initialize the traffic environment

env.start() # start the connection with SUMO

for episode in range(human_learning_episodes): # human learning 
    env.step()

env.mutation() # some human agents transition to AV agents

collector = SyncDataCollector(env, policy, ...)  # collects experience by running the policy in the environment (TorchRL)

# training of the autonomous vehicles; human agents follow fixed decisions learned in their learning phase
for tensordict_data in collector:
        
    # update the policies of the learning agents
    for _ in range(num_epochs):
      subdata = replay_buffer.sample()
      loss_vals = loss_module(subdata)

      optimizer.step()
    collector.update_policy_weights_()

policy.eval() # set the policy into evaluation mode

# testing phase using the already trained policy
num_episodes = 100
for episode in range(num_episodes):
    env.rollout(len(env.machine_agents), policy=policy)
 
env.plot_results() # plot the results
env.stop_simulation() # stop the connection with SUMO

Documentation

• Tutorials:
  • Quickstart.
  • Medium network and AVs behaviors.
  • Big network and independent AV agents.

Installation

• Prerequisite: Make sure you have SUMO installed in your system. This procedure should be carried out separately, by following the instructions provided here.
• Option 1: Install the latest stable version from PyPI:

    pip install routerl
  

• Option 2: Clone this repository for latest version, and manually install its dependencies:

    git clone https://github.com/COeXISTENCE-PROJECT/RouteRL.git
    cd RouteRL
    pip install -r requirements.txt
  

Reproducibility capsule

We have an experiment script encapsulated in a CodeOcean capsule. This capsule allows demonstrating RouteRL's capabilities without the need for SUMO installation or dependency management.

1. Visit the capsule link.
2. Create a free CodeOcean account (if you don’t have one).
3. Click Reproducible Run to execute the code in a controlled and reproducible environment.

Citation

If you use RouteRL in your research, please cite the following paper:

@article{routerl,
  title = {RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehicles},
  journal = {SoftwareX},
  volume = {31},
  pages = {102279},
  year = {2025},
  issn = {2352-7110},
  doi = {https://doi.org/10.1016/j.softx.2025.102279},
  url = {https://www.sciencedirect.com/science/article/pii/S2352711025002468},
  author = {Ahmet Onur Akman and Anastasia Psarou and Łukasz Gorczyca and Zoltán György Varga and Grzegorz Jamróz and Rafał Kucharski}
}

Credits

RouteRL is part of COeXISTENCE (ERC Starting Grant, grant agreement No 101075838) and is a team work at Jagiellonian University in Kraków, Poland by: Ahmet Onur Akman and Anastasia Psarou (main contributors) supported by Grzegorz Jamroz, Zoltán Varga, Łukasz Gorczyca, Michał Hoffmann, Błażej Torbus, Mikołaj Rams and others, within the research group of Rafał Kucharski.