smarts 0.4.2

Scalable Multi-Agent RL Training School

SMARTS

SMARTS (Scalable Multi-Agent RL Training School) is a simulaton platform for reinforcement learning and multi-agent research on autonomous driving. Its focus is on realistic and diverse interactions. It is part of the XingTian suite of RL platforms from Huawei Noah's Ark Lab.

Multi-Agent experiment as simple as...

import gym

from smarts.core.agent_interface import AgentInterface, AgentType
from smarts.core.agent import AgentSpec, AgentPolicy

class Policy(AgentPolicy):
    def act(self, obs):
        return "keep_lane"

agent_spec = AgentSpec(
    interface=AgentInterface.from_type(AgentType.Laner, max_episode_steps=None),
    policy_builder=Policy,
)

agent_specs = {
    "Agent-007": agent_spec,
    "Agent-008": agent_spec,
}

env = gym.make(
    "smarts.env:hiway-v0",
    scenarios=["scenarios/loop"],
    agent_specs=agent_specs,
    visdom=True,
)

agents = {
    agent_id: agent_spec.build_agent()
    for agent_id, agent_spec in agent_specs.items()
}
observations = env.reset()

for _ in range(1000):
    agent_actions = {
        agent_id: agents[agent_id].act(agent_obs)
        for agent_id, agent_obs in observations.items()
    }
    observations, _, _, _ = env.step(agent_actions)

Setup

# For Mac OS X users, make sure XQuartz is pre-installed as SUMO's dependency

# git clone ...
cd <project>

# Follow the instructions given by prompt for setting up the SUMO_HOME environment variable
./install_deps.sh

# verify sumo is >= 1.5.0
# if you have issues see ./doc/SUMO_TROUBLESHOOTING.md
sumo

# setup virtual environment; presently only Python 3.7.x is officially supported
python3.7 -m venv .venv

# enter virtual environment to install all dependencies
source .venv/bin/activate

# upgrade pip, a recent version of pip is needed for the version of tensorflow we depend on
pip install --upgrade pip

# install [train] version of python package with the rllib dependencies
pip install -e .[train]

# make sure you can run tests (and verify they are passing)
make test

# then you can run a scenario, see following section for more details

Running

We use supervisord to run SMARTS together with it's supporting processes. To run the default example simply build a scenario and start supervisord:

# build scenarios/loop
scl scenario build --clean scenarios/loop

# start supervisord
supervisord

With supervisord running, visit http://localhost:8081/ in your browser to view your experiment.

See ./envision/README.md for more information on Envision, our front-end visualization tool.

Several example scripts are provided under SMARTS/examples, as well as a handful of scenarios under SMARTS/scenarios. You can create your own scenarios using the Scenario Studio. Here's how you can use one of the example scripts with a scenario.

# Update the command=... in ./supervisord.conf
#
# [program:smarts]
# command=python examples/single_agent.py scenarios/loop
# ...

CLI tool

SMARTS provides a command-line tool to interact with scenario studio and Envision.

Usage

scl COMMAND SUBCOMMAND [OPTIONS] [ARGS]...

Commands:

• envision
• scenario
• zoo

Subcommands of scenario:

• build-all: Generate all scenarios under the given directories
• build: Generate a single scenario
• clean: Clean generated artifacts

Subcommands of envision:

• start: start envision server

Subcommands of zoo:

• zoo: Build an agent, used for submitting to the agent-zoo

Examples:

# Start envision, serve scenario assets out of ./scenarios
scl envision start --scenarios ./scenarios

# Build all scenario under given directories
scl scenario build-all ./scenarios ./eval_scenarios

# Rebuild a single scenario, replacing any existing generated assets
scl scenario build --clean scenarios/loop

# Clean generated scenario artifacts
scl scenario clean scenarios/loop

Interfacing with Gym

See the provided ready-to-go scripts under the examples/ directory.

Contributing

Please read Contributing

Building Docs Locally

Assuming you have run pip install .[dev].

make docs

python -m http.server -d docs/_build/html
# Open http://localhost:8000 in your browser

Extras

Visualizing Agent Observations

If you want to easily visualize observations you can use our Visdom integration. Start the visdom server before running your scenario,

visdom
# Open the printed URL in your browser

Using Docker

If you're comfortable using docker or are on a platform without suitable support to easily run SMARTS (e.g. an older version of Ubuntu) you can run the following,

docker build -t smarts .
docker run --rm -it -v $(PWD):/src -p 8081:8081 smarts
# <press enter>

cd /src
pip install -e .[train]

# Run Envision server in the background
$ scl envision start -s ./scenarios -p 8081 &
$ python examples/single_agent.py scenarios/loop

# On your most machine visit http://localhost:8081 to see the running simulation in
# Envision.

SUMO Troubleshooting

• If you are having issues see: SETUP and SUMO TROUBLESHOOTING
• If you wish to find binaries: SUMO Download Page
• If you wish to compile from source see: SUMO Build Instructions
• If you build from the git repository use: SUMO version 1.7.0 or higher