simforge-gym

SimForge Gym: train driving models in SimForge on your own machine. Today that is reinforcement learning; supervised fine-tuning recipes are coming. simforge-gym gives you Gymnasium environments over the SimForge runtime, task sets on the released SimForge maps, training on SimForge datasets, and a PPO example. Apache-2.0. Dependencies: numpy and gymnasium; the PPO example adds torch (pip install 'simforge-gym[ppo]').

The episodes run in the SimForge runtime, a closed program you install after signing in; this package is the open client. It never imports runtime code and never substitutes another simulator: no runtime, no episode. Training runs locally; there is no cloud RL.

simforge-gym 0.2.0 is the package formerly published as simforge-rl (frozen at 0.1.4): pip install simforge-gym and import simforge_gym, with the same API. Scenario-set and run files now carry simforge-gym.* schema ids.

On Windows, the package requires a runtime that reports capabilities.envSocketTransport: "named-pipe" (the older 0.1.0+bfd484b7727b1cfd runtime cannot run local RL). Update with simforge runtime install; older runtimes fail with runtime_upgrade_required. The same env-serve protocol uses secure same-user named pipes on Windows and Unix sockets on macOS/Linux.

Quickstart (CPU only)

State observations need no GPU. With the simforge CLI installed:

simforge login                                                 # a browser approves the device code
simforge runtime install --accept-license                      # the closed runtime, under ~/.local/share/simforge/runtime
simforge maps pull richmond-field-station@v9 --accept-license  # a released map
pip install 'simforge-gym[ppo]'
python -m simforge_gym.examples.ppo --tasks richmond-field-station --runtime --num-envs 8 --total-timesteps 200000 --out run/

run/ then holds curve.csv, curve.svg, metadata.json and policy.pt. --runtime runs each vector worker's episodes in its own simforge-runtime serve process; without it the example starts simforge env serve processes through the CLI instead. Both give the same observations and rewards.

In Python:

from simforge_gym import RuntimeSessions, ScenarioSet, make_vector_env

tasks = ScenarioSet.bundled("richmond-field-station")          # 24 tasks: 15 templates at verified sites
with RuntimeSessions(tasks, "--no-sensors", workdir="work") as sessions:
    envs = make_vector_env(tasks, sessions.transports(), num_envs=8)
    obs, info = envs.reset(seed=0)
    obs, reward, terminated, truncated, info = envs.step(envs.action_space.sample())   # [target_speed_mps, target_accel_mps2] per env

Task sets on the released maps

bundled_sets() names the task sets shipped with the package: belmont-research-center, easterbrook-discovery-school, el-camino-road, richmond-field-station and yale-street, 24 to 29 tasks each. Each is a weighted set of scenario templates (car following, cut-ins, lead braking, crossing pedestrians and cyclists, unprotected turns, red-light runners, vehicles pulling out, ...), each at the sites of that map where it compiles and its authored episode runs, with every template weighted equally. A set is pinned to the OpenDRIVE digest of the map version it was verified on, so its episodes run on exactly that installed map: pull NAME@VERSION as the set's map says. evidence/task-sets/report.json records every template, site and verification episode. Site ids come from the map pack's derived road data, so a republished pack comes with rebuilt sets (scripts/build_task_sets.py); a site the installed pack does not have fails with no_site rather than moving the task elsewhere.

Every actor in the templates is in the public CARLA asset pack. The urban mixed template uses carla.vehicle_fuso_mitsubishi for its oncoming bus; vehicle.bus is the full-size procedural model and is not in that pack.

Train on a dataset

tasks = ScenarioSet.from_dataset("data/my-dataset", split="train")   # a pulled folder
tasks = ScenarioSet.from_dataset("usds_01JA...", split="train")      # your hosted dataset, pulled with `simforge datasets pull`

simforge datasets pull usds_01JA... --include scenarios --out data/my-dataset
python -m simforge_gym.examples.ppo --dataset data/my-dataset --split train --runtime --max-open 16 --eval-limit 8 --out run/

Both dataset formats work: the public SimForge dataset (simforge.public-dataset/v1) and hosted datasets pulled as files (simforge.dataset-files/v1). Every file is checked against its recorded digest and laid out once; a pulled scenario's episode is exactly the input its simulation ran, and every episode runs on the installed map the scenario's OpenDRIVE digest names. --max-open keeps at most that many sessions open per worker (least recently used close first), so a split of thousands of scenarios trains with a bounded number of sessions.

Camera observations

observation="sensors" returns rendered camera passes keyed "<sourceId>/<pass>" (rgb uint8 HxWx4, depth float32 HxW) and nothing else: the policy sees no engine state, object ids, reward terms or events. Rendering needs a supported GPU and an installed map's 3D pack. simforge_gym/examples/rigs/alpamayo-fisheye.car.rig.json is an Alpamayo-style rig: a 120-degree f-theta (fisheye) front camera and a 30-degree front tele camera, 512x384. Its lens is a generic equidistant calibration, not one taken from a real vehicle.

simforge env serve ws --socket cam.sock --rig alpamayo-fisheye.car.rig.json --observation-profile sensors \
  --passes rgb,depth --actor-closure "$(basename ~/.local/share/simforge/runtime/share/actors/closures/*.json .json)"

env = SimForgeEnv(SocketTransport("cam.sock"), observation="sensors")
obs, info = env.reset(options={"episode_seed": 1})
frame = obs["camera_front_wide_120fov/rgb"]          # (384, 512, 4) uint8

If the policy drives the ego off the map, the runtime has no camera pose for that step and renders nothing. The step then returns the last rendered camera observation, truncated=True, and the runtime's sensor_host_absent warning (naming the absent actor) in info["render_warnings"]. Reset to start the next episode. Only a reset whose ego is already out of the world raises, because there is no earlier frame to return.

ws is a scenario workspace, for example template_workspace(...) of a bundled task. simforge rig show alpamayo-fisheye.car.rig.json reports each camera's field of view; the installed runtime plans the f-theta lens.

Rewards

reward_fn(native_reward, info) -> float replaces the reward; info["native_reward"] keeps the runtime's. WeightedTerms({"progress": 1.0, "proximity": 2.0, "comfort": 1.0}) reweights the runtime's reward terms and EventPenalty({"collision": 10.0}) subtracts per engine event; both need a state observation mode.

The PPO result

evidence/ppo-scenario-set is a recorded PPO run on the bundled Richmond Field Station set through the distributable runtime, with its learning curve, configuration and a same-seed repeat.

The policy sees pose-free ego state and the eight nearest object rows and outputs a speed and acceleration setpoint. Each evaluation also runs the scenarios' authored ego control on the same tasks as a baseline. python -m simforge_gym.examples.ppo --help lists every flag.

More

• API reference: environments, task sets, datasets, the runtime protocol client (episodes, CPU batches of up to 64 episodes, persistent worlds) and the PPO flags.
• RL methodology: what the environment simulates, how it is evaluated, and what has been measured.

Development

uv run --project clients/gym --extra dev python -m pytest -q clients/gym/tests
python3 scripts/sdk/toolkit-release.py build --ref HEAD --out "$TMPDIR/wheels"   # reproducible wheels

SIMFORGE_BIN and SIMFORGE_GYM_TEST_MAP_DIR (Richmond Field Station v9, in a directory named richmond-field-station) enable the live runtime tests; SIMFORGE_GYM_TEST_SENSOR_MAP and SIMFORGE_GYM_TEST_ACTOR_CLOSURE add the camera test. Without them those tests are skipped and say so. scripts/build_task_sets.py rebuilds the bundled task sets from installed map packs.