Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Tutorials Documentation Python 3.9–3.13 PyPI Version Publication DOI License Open in Code Ocean

Welcome to the future of urban routing.

Multi-agent reinforcement learning framework for modeling and simulating the collective route choices of humans and autonomous vehicles.


RouteRL is a multi-agent reinforcement learning framework that integrates RL-based collective route choice with a microscopic traffic simulation, SUMO, facilitating the testing and development of efficient route choice strategies. The proposed framework simulates the daily route choices of driver agents in a city, including two types:

  • human drivers, emulated using discrete choice models,
  • and AVs, modeled as MARL agents optimizing their policies for a predefined objective.

RouteRL aims to advance research in MARL, traffic assignment problems, social reinforcement learning, and human-AI interaction for transportation applications.

  • The main class is TrafficEnvironment and is a PettingZoo AEC API environment.
  • There are two types of agents in the environment and are both represented by the BaseAgent class.
    • Human drivers are simulated using human route-choice behavior from transportation research.
    • Automated vehicles (AVs) are the RL agents that aim to optimize their routes and learn the most efficient paths.
  • RouteRL is compatible with popular RL libraries such as TorchRL.

For overview see the paper and for more details, check the documentation online.

RouteRL usage and functionalities at a glance

The following is a simplified code of a possible standard MARL algorithm implementation via TorchRL.

env = TrafficEnvironment(seed=42, **env_params) # initialize the traffic environment

env.start() # start the connection with SUMO

for episode in range(human_learning_episodes): # human learning 
    env.step()

env.mutation() # some human agents transition to AV agents

collector = SyncDataCollector(env, policy, ...)  # collects experience by running the policy in the environment (TorchRL)

# training of the autonomous vehicles; human agents follow fixed decisions learned in their learning phase
for tensordict_data in collector:
        
    # update the policies of the learning agents
    for _ in range(num_epochs):
      subdata = replay_buffer.sample()
      loss_vals = loss_module(subdata)

      optimizer.step()
    collector.update_policy_weights_()

policy.eval() # set the policy into evaluation mode

# testing phase using the already trained policy
num_episodes = 100
for episode in range(num_episodes):
    env.rollout(len(env.machine_agents), policy=policy)
 
env.plot_results() # plot the results
env.stop_simulation() # stop the connection with SUMO

Documentation

Installation

  • Prerequisite: Make sure you have SUMO installed in your system. This procedure should be carried out separately, by following the instructions provided here.
  • Option 1: Install the latest stable version from PyPI:
      pip install routerl
    
  • Option 2: Clone this repository for latest version, and manually install its dependencies:
      git clone https://github.com/COeXISTENCE-PROJECT/RouteRL.git
      cd RouteRL
      pip install -r requirements.txt
    

Reproducibility capsule

We have an experiment script encapsulated in a CodeOcean capsule. This capsule allows demonstrating RouteRL's capabilities without the need for SUMO installation or dependency management.

  1. Visit the capsule link.
  2. Create a free CodeOcean account (if you don’t have one).
  3. Click Reproducible Run to execute the code in a controlled and reproducible environment.

Citation

If you use RouteRL in your research, please cite the following paper:

@article{routerl,
  title = {RouteRL: Multi-agent reinforcement learning framework for urban route choice with autonomous vehicles},
  journal = {SoftwareX},
  volume = {31},
  pages = {102279},
  year = {2025},
  issn = {2352-7110},
  doi = {https://doi.org/10.1016/j.softx.2025.102279},
  url = {https://www.sciencedirect.com/science/article/pii/S2352711025002468},
  author = {Ahmet Onur Akman and Anastasia Psarou and Łukasz Gorczyca and Zoltán György Varga and Grzegorz Jamróz and Rafał Kucharski}
}

Credits

RouteRL is part of COeXISTENCE (ERC Starting Grant, grant agreement No 101075838) and is a team work at Jagiellonian University in Kraków, Poland by: Ahmet Onur Akman and Anastasia Psarou (main contributors) supported by Grzegorz Jamroz, Zoltán Varga, Łukasz Gorczyca, Michał Hoffmann, Błażej Torbus, Mikołaj Rams and others, within the research group of Rafał Kucharski.

Metadata

Release files for routerl-nightly 2.0.0.dev261110091236

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for routerl-nightly 2.0.0.dev261110091236
File Size Uploaded Provenance Actions
routerl_nightly-2.0.0.dev261110091236.tar.gz 2.3 MB With provenance Details for routerl_nightly-2.0.0.dev261110091236.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for routerl-nightly 2.0.0.dev261110091236
File Interpreter ABI Platform Actions
routerl_nightly-2.0.0.dev261110091236-py3-none-any.whl Python 3 none any Details for routerl_nightly-2.0.0.dev261110091236-py3-none-any.whl

Total release size: 4.6 MB

Release files / routerl_nightly-2.0.0.dev261110091236.tar.gz

Download URL routerl_nightly-2.0.0.dev261110091236.tar.gz
Size 2.3 MB
Tags Source
SHA-256 checksum
How to use checksums
e823d3d7d54b8c907dde8783697ec56f2de3cf8839e081885f76a7afbe503fb5
BLAKE2b-256 checksum
How to use checksums
967669c560db741b49723d4af273a700953f1b5c622622d5f835d2f335a642fe
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

Release files / routerl_nightly-2.0.0.dev261110091236-py3-none-any.whl

Download URL routerl_nightly-2.0.0.dev261110091236-py3-none-any.whl
Size 2.4 MB
Tags Python 3
SHA-256 checksum
How to use checksums
741129eb766cc744ea01c4115fa4cf919123dc70c2c373023bbe9330db7765f1
BLAKE2b-256 checksum
How to use checksums
f3e59bb5d28fd784c1eda15501f9c8d81428be9ee1077e0eaab20ff9749ff8ee
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

Security for routerl-nightly 2.0.0.dev261110091236

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for routerl_nightly-2.0.0.dev261110091236.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 10, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page