Skip to main content

inspect-robots-yam

Run Inspect Robots evals on real I2RT YAM bimanual arms driven by MolmoAct2.

Status: alpha CI PyPI License: MIT Coverage Docs coverage Built on Inspect Robots

Note: This project is in early development. The API may change between releases, so pin a version before depending on it.

Inspect Robots has two swappable inputs: a Policy (the VLA brain) and an Embodiment (the robot body + world). This package provides both for the YAM + MolmoAct2 stack, so any embodiment-agnostic Inspect Robots task (e.g. all of KitchenBench) runs on real arms:

  • molmoact2 policy: a thin client for MolmoAct2's first-party bimanual-YAM /act server (the model owns the GPU + weights in its own process).
  • yam_arms embodiment: the I2RT joint-position driver, with a hard safety clamp, operator-in-the-loop success, and self-paced control.

Both declare the same 14-D joint-position contract (2 arms × [6 joints + gripper], cameras top/left/right, packed joint_pos state), so Inspect Robots's compatibility check passes with zero errors and zero warnings. This is verifiable before any motion.

inspect-robots run --task kitchenbench/pour_pasta --policy molmoact2 --embodiment yam_arms

Note: cameras are configured with three plain device paths (top/left/right_cam_device), so the whole rig is drivable from config.ini or -E key=value flags with no custom code. A Python camera_reader remains available for exotic camera stacks. With neither configured, yam_arms fails fast with a ConfigError at reset(), before any driver connect or motion.

Install (on the robot/GPU machine)

uv pip install inspect-robots-yam
# The i2rt driver is git-only and not on PyPI. Install it directly:
uv pip install "i2rt @ git+https://github.com/i2rt-robotics/i2rt"

The base package includes the /act transport and builtin OpenCV camera reader. Only i2rt, the I2RT YAM arm driver required for real hardware, needs the separate git install. The camera reader depends on opencv-python-headless; if your environment also carries opencv-python, the two share the cv2 module and the last one installed wins.

Then download the model weights (needs a Hugging Face token) and start the server, from the MolmoAct2 repo:

huggingface-cli download allenai/MolmoAct2-BimanualYAM
python examples/yam/host_server_yam.py          # serves /act on :8202

Preflight: prove compatibility before any motion

Check dims, semantics, cameras, and state keys:

inspect-robots-yam-preflight

Also check a specific task's scenes are realizable:

inspect-robots-yam-preflight --task kitchenbench/pour_pasta

Affirm that no motion will occur:

inspect-robots-yam-preflight --dry-run

A green preflight means action dim (14), control mode (joint_pos), cameras, and state keys all line up. It does not prove the joint values are interpreted the same way. See Safety below.

Run on hardware

Write your defaults once. The interactive wizard interviews this plugin's declared devices (three cameras and both arms' CAN channels) with live probes, including unplug-to-identify:

uv run inspect-robots setup

Or write the file yourself, replacing the three camera paths with your rig's V4L2 color nodes (use stable /dev/v4l/by-id/... or udev-symlink paths; bare /dev/videoN numbers reshuffle on every replug):

mkdir -p ~/.config/inspect-robots && cat > ~/.config/inspect-robots/config.ini <<'EOF'
[defaults]
policy = molmoact2
embodiment = yam_arms
scorer = success_at_end    # scores the operator's y/N answer at episode end
max_steps = 1200           # 120 s at 10 Hz
rerun = true               # live viewer of cams/state/actions (inspect-robots[rerun])
store_frames = true        # keep the policy's camera frames per run

[embodiment.args]
top_cam_device = /dev/v4l/by-id/YOUR-TOP-CAM
left_cam_device = /dev/v4l/by-id/YOUR-LEFT-CAM
right_cam_device = /dev/v4l/by-id/YOUR-RIGHT-CAM
EOF

Then tell the robot what to do:

uv run inspect-robots "place the fork on the plate"

The attended flow: position the scene, press Enter to start, press any key to end the episode, answer y/N to score.

For exotic camera stacks (or full programmatic control), the Python API takes a custom camera_reader returning {"top_cam", "left_cam", "right_cam": HxWx3 uint8}:

from inspect_robots import eval
from inspect_robots.approver import ClampApprover
from inspect_robots_yam import MolmoAct2Policy, YAMEmbodiment, YamConfig

emb = YAMEmbodiment(YamConfig(left_channel="can0", right_channel="can1"),
                    camera_reader=my_camera_reader)
pol = MolmoAct2Policy(server_url="http://127.0.0.1:8202")

(log,) = eval("kitchenbench/pour_pasta", pol, emb,
              approver=ClampApprover(emb.info.action_space))  # defense in depth
print(log.status, log.results.metrics)

At each episode end the embodiment asks the operator (y/N); a yes records termination_reason="success", which KitchenBench's task_success scorer reads. The operator prompts need an interactive terminal: a dead stdin raises EmbodimentFault (the framework's always-halt path). For runs with no operator, set YamConfig(unattended=True) (CLI: -E unattended=true): all operator prompts are skipped and every episode runs to max_steps, scoring as a failure.

Drive the arms with an LLM (agent mode)

With the inspect-robots-agent plugin installed, a frontier LLM can drive the arms directly: it sees the cameras and the labeled 14-D state, and moves joints by name (left_j0..left_gripper, right_j0..right_gripper) through smooth, approver-checked motions.

uv pip install inspect-robots-agent inspect-robots-yam
inspect-robots config set embodiment yam_arms     # once, per machine
export ANTHROPIC_API_KEY=sk-ant-...

# Cameras come from the builtin reader: set the three *_cam_device paths in
# ~/.config/inspect-robots/config.ini (see Quickstart above) or pass them as
# -E flags per run.
inspect-robots "place the fork on the plate" --policy agent \
    -P model=anthropic/claude-fable-5

Safety guardrails (a bounds clamp plus a per-step delta limit derived from the declared action space) are wired in by default for every CLI run; turning them off requires an explicit --disable-guardrails.

Warning: Before any unattended agent run, verify on your rig that the arms hold position while the LLM thinks (seconds between action chunks). Run the bundled check per arm and per mode, arms mid-workspace, e-stop in hand:

inspect-robots-yam-holdcheck can_left --zero-gravity true
inspect-robots-yam-holdcheck can_right --zero-gravity true

(Channel names match your rig's CAN interfaces; can0/can1 on default setups.) PASS in the mode you run agents in closes the verification. The default zero_gravity_mode=true puts the i2rt driver in a gravity-compensated, compliant mode; if it drifts but --zero-gravity false holds, run agents with -E zero_gravity_mode=false. If both drift, file an issue with the numbers. Keep a hand on the e-stop for the first runs.

Set a resting pose so runs end with a gentle 3-second park instead of the arms going limp mid-air (pose fields accept comma-separated values from the CLI and config.ini):

[embodiment.args]
rest_pose = -0.002,0.002,0.002,-0.089,0.007,-0.026,1.0,-0.006,0.002,0.001,-0.087,-0.007,-0.019,1.0

In delta mode (-E joints_are_delta=true) the declared action space is the per-step displacement box (YamConfig.step_limits, default 0.2 rad per joint and a full gripper stroke per step); the absolute joint limits still clamp the summed command inside the embodiment as a backstop. A delta-configured rig must be paired with a delta-declaring policy (-P joints_are_delta=true for molmoact2); a mismatch fails the compatibility check before any motion.

Safety

  • Hard clamp backstop. Every command is clipped to YamConfig.joint_low/high inside step(), independent of any Inspect Robots Approver: unclamped model outputs can never reach the motors. Set the arm slots to your real YAM joint limits (the defaults are conservative placeholders: joints ±π, gripper 0–1). But note the limits are in policy units per the table below: gripper slots 6 and 13 stay normalized 0–1, only slots 0–5 and 7–12 are radians.
  • Use ClampApprover on hardware for a second layer.
  • Zero-gravity handoff jump. The arms connect in zero-gravity mode by default (YamConfig(zero_gravity_mode=True), passed through to the i2rt driver). Homing and rest-pose motions ramp at control_hz, but the first policy action is still a stiff PD command that can jump from wherever the arm ended up. Nothing bounds the per-step joint delta yet (tracked as a known issue); stand clear when the episode starts, and set home_pose so episodes begin from your checkpoint's trained start state.
  • Absolute vs. delta joints: verify first. MolmoAct2's YAM actions are treated as absolute joint targets by default. If your checkpoint emits deltas, set YamConfig(joints_are_delta=True) (the embodiment converts to absolute internally so the declared joint_pos stays honest). Inspect Robots's compat check cannot tell these apart: confirm with --dry-run and a single slow jog before running a task.
  • Gripper polarity/trim. The i2rt driver already exposes the YAM gripper as normalized 0–1 in both directions, so the defaults (gripper_open=0.0, gripper_closed=1.0) are an identity map and correct for standard grippers. YamConfig(gripper_open=..., gripper_closed=...) is a polarity/trim remap over that already-normalized range. Its main use is a gripper wired with inverted polarity (gripper_open=1.0, gripper_closed=0.0). The remap is a bijection: commands are de-normalized on the way out and observations are re-normalized on the way back, so the model always sees 0–1. Warning: values outside [0, 1] are forwarded on a path i2rt does not clip. Avoid them unless you have verified your firmware's behavior.

Configuration

Units: every 14-D vector uses the same layout

joint_low/joint_high, home_pose, rest_pose, actions, and the observed joint_pos state all use policy units:

Slots Meaning Unit
0–5, 7–12 left / right arm revolute joints radians
6, 13 left / right gripper normalized 0–1 (0 = open, 1 = closed)

Hardware gripper units (via gripper_open/gripper_closed) exist only at the driver boundary; nothing you configure here is in hardware gripper units.

YamConfig: left_channel, right_channel, gripper_type (i2rt GripperType enum name, e.g. LINEAR_4310; grippers only: NO_GRIPPER/YAM_TEACHING_HANDLE would break the 14-D packing and are rejected), control_hz, cam_height/width, joint_low/high, home_pose (reset ramps here smoothly over rest_secs rather than jumping), rest_pose (close ramps here before torque is released, so the arms never fall; default None keeps the old release-in-place behavior), rest_secs (ramp duration, default 3.0), gripper_open/closed, joints_are_delta, zero_gravity_mode (default True; see Safety), unattended (default False; skip operator prompts), top/left/right_cam_device (V4L2 paths for the builtin camera reader; all three or none), max_steps_hint (display-only horizon for the operator status line; bounds nothing). MolmoActConfig: server_url, endpoint, num_steps (the wire field: the server's flow-matching denoising steps, not the chunk length), action_horizon (the checkpoint's advertised chunk length, 30 for the bimanual YAM tag; metadata only), timeout_s, camera_order, state_key, cam_height/width.

Scalar knobs are settable from the CLI: inspect-robots run -P server_url=http://gpu:8202 -E left_channel=can0 ....

Development

Dependency changes: after editing dependencies in pyproject.toml, run uv lock and commit the updated lockfile. CI installs with uv sync --locked and fails with "the lockfile needs to be updated" if you forget. Day-to-day conventions (PR-only main, the required ci-ok check, one-click releases) are documented in CLAUDE.md.

uv venv && uv pip install -e ".[dev]"     # inspect_robots + kitchenbench from PyPI
uv run pre-commit install
uv run pytest --cov                        # 100% coverage required
uv run ruff check . && uv run mypy

Every public module, class, and function needs a docstring, enforced by Ruff D1; state the contract instead of restating the symbol name.

The whole suite runs with no hardware, no server, and no stdin: the i2rt driver, cameras, the /act transport, the clock, and operator I/O are all injected. The default hardware seams are excluded from coverage (# pragma: no cover).

Citation

If you use Inspect Robots YAM in your research, please cite it:

@software{inspect-robots-yam,
  author  = {Robocurve},
  title   = {Inspect Robots YAM: Adapters for I2RT YAM bimanual arms},
  year    = {2026},
  url     = {https://github.com/robocurve/inspect-robots-yam},
  version = {0.3.0},
  license = {MIT}
}

License

MIT

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

inspect_robots_yam-0.8.0.tar.gz (123.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

inspect_robots_yam-0.8.0-py3-none-any.whl (34.1 kB view details)

Uploaded Python 3

File details

Details for the file inspect_robots_yam-0.8.0.tar.gz.

File metadata

  • Download URL: inspect_robots_yam-0.8.0.tar.gz
  • Upload date:
  • Size: 123.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for inspect_robots_yam-0.8.0.tar.gz
Algorithm Hash digest
SHA256 7a18252c30408de9ae9ad4601185b746aa861b62dcfe1b2577d1996ae2e80b58
MD5 c15d757b5f224a084f66802f252582dc
BLAKE2b-256 d9ed090d2d73e636bf798577bdbf2e6f3f51354046526c941440537e275f37e1

See more details on using hashes here.

Provenance

The following attestation bundles were made for inspect_robots_yam-0.8.0.tar.gz:

Publisher: release.yml on robocurve/inspect-robots-yam

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file inspect_robots_yam-0.8.0-py3-none-any.whl.

File metadata

File hashes

Hashes for inspect_robots_yam-0.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 5809a95201757c8f610f19d09909d023c2fd8ef5f97d38382def1b6904c055a2
MD5 c2aa0c6478d86f47aeff48649af8e626
BLAKE2b-256 a002897a57c8837a1127e1c73821b1f31017569495779dbf9db6e01ac60cfb81

See more details on using hashes here.

Provenance

The following attestation bundles were made for inspect_robots_yam-0.8.0-py3-none-any.whl:

Publisher: release.yml on robocurve/inspect-robots-yam

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.36.0

2 files

0.35.1

2 files

0.35.0

2 files

0.34.0

2 files

0.33.0

2 files

0.32.0

2 files

0.31.0

2 files

0.30.0

2 files

0.29.0

2 files

0.28.0

2 files

0.27.0

2 files

0.26.0

2 files

0.25.0

2 files

0.24.0

2 files

0.23.0

2 files

0.22.0

2 files

0.21.1

2 files

0.21.0

2 files

0.20.0

2 files

0.19.0

2 files

0.18.0

2 files

0.17.0

2 files

0.16.0

2 files

0.15.1

2 files

0.15.0

2 files

0.14.0

2 files

0.13.0

2 files

0.12.0

2 files

0.11.0

2 files

0.10.0

2 files

0.9.1

2 files

0.9.0

2 files

0.8.2

2 files

0.8.1

2 files

This release

0.8.0 This release

2 files

0.7.0

2 files

0.6.0

2 files

0.5.0

2 files

0.4.0

2 files

0.3.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page