OpenRUA

Let Your Claude Code or Codex Control Any Robot, Real or Simulated

Through the standard ROS 2 CLI and client library, without relying on any VLA model.

Task sentence here, model here, outcome here. More in docs/demos.md.

A robot-use agent uses a robot just as a computer-use agent uses a computer. OpenRUA is the open harness for one: type openrua run panda "pick up the red cube" and Claude Code or Codex opens in a terminal on the robot's ROS 2 graph, lists the topics, reads the docs in its workspace, writes a script with rclpy, runs it, and checks the camera.

OpenRUA gives you one command for three things:

• Play in simulation. openrua run brings up a Franka Panda in MuJoCo; the agent drives it the same way it would a real one.
• Put an agent on your robot. Draft its file from the robot's live graph, finish the TODO lines, openrua run <name>. See docs/your-own-robot.md.
• Run experiments. openrua bench plays a benchmark across tasks and seeds with a fresh sandbox per trial and archives every command the agent ran. See docs/running-experiments.md.

OpenRUA turns your robot into a coding project: your agent explores it like a live codebase, pulls sensor streams into files for reading, and runs commands and programs to move it.

Start playing with your robot like you code a project :)

Quick start

Install, choose a simulator and an agent once, name a robot, run:

pip install git+https://github.com/terminalworld/OpenRUA
openrua config set --sim robosuite --agent claude-code   # your defaults
openrua build                                       # the three images, once
openrua install --sim robosuite                     # the simulator's checkout and venv, once
openrua run panda "pick up the red cube"

The robot comes up on its ROS 2 graph, the agent opens on its terminal with that sentence, and the robot powers off when you leave; run first prints which robot, scene and agent it picked.

openrua doctor tells you what is missing before the first run (Docker, the three images, the simulator install, an agent login); the details are in docs/install.md.

Choosing what to run

• The world. Without --bench the scene is the simulator's own (robosuite: a table and a cube). --bench libero_pro loads a benchmark's world instead, a LIBERO kitchen with a bowl, a plate, a wine bottle, a drawer and a stove; --task-suite and --task-id pick a scene inside it.
• The agent. --agent codex opens Codex; --model picks the model.
• Once or every time. Whatever config set stored can also be given on the command line: openrua run panda --sim robosuite --bench libero_pro "pick up the bowl". openrua robots, openrua simulators, openrua benchmarks and openrua agents list the choices; openrua benchmarks libero_pro lists one benchmark's suites and tasks.
• A robot that stays up. The same three steps as separate commands: openrua up, then openrua agent "..." in a second terminal, then openrua down.

How it works

 your terminal                       the robot (real or simulated)
 ┌─────────────────────────┐        ┌──────────────────────────────┐
 │ openrua run             │        │ ROS 2 graph                  │
 │  └─ Claude Code / Codex │  DDS   │  /joint_states  /tf  /camera │
 │      in a sandbox with  │◄──────►│  FollowJointTrajectory       │
 │      ros2 · rclpy · docs│        │  GripperCommand  MoveIt      │
 └─────────────────────────┘        └──────────────────────────────┘

• The interface is the robot's own. The agent sees the topics, actions and services the robot exposes, plus machine.yaml (joints, limits, frames, ports) and four short docs. It never sees OpenRUA.
• The sandbox is a plain Ubuntu + ROS 2 container with the agent installed, a workspace mounted, and a whitelist proxy as its only way out (the model API; nothing else).
• A robot is a file (openrua/configs/robots/): the facts true of it wherever it runs (joints, limits, frames, gripper, ports). Your real robot is the same kind of file with a machine: section that says how to reach its graph, passed by path.
• A simulator is a file (openrua/configs/simulators/): the engine, its install, its native scene, and how it drives each robot it embodies.
• A benchmark is a file (openrua/configs/benchmarks/): which robot and simulator, which suites and init states to load, how a trial runs and stops. openrua bench runs trials, checks every promise the workspace docs make before the agent starts, and records each trial with full provenance.
• Everything is checked against one schema (openrua config schema): a misspelled key in any file is an error, never a silent no-op. Your defaults live in ~/.openrua/config.yaml.

Supported robots

A robot is what is true of it wherever it runs: joints, limits, frames, gripper, ports, planner. Which simulator embodies it, and what surrounds it, come from the other two kinds of file.

openrua robots prints this list from the files on disk, yours included.

Supported simulators

A simulator file knows the engine and how it drives each robot it embodies; it knows no benchmark. Its install (a venv under ~/.openrua/simulators/) and the benchmarks' own are described in docs/simulation.md.

Supported benchmarks

A benchmark names its robot and simulator and brings its own world: install: (its venv and ROS distro) and scenes: (scene cameras, and robot embodiments its assets add). openrua benchmarks prints this list; openrua bench --config <name> runs one.

Supported agents

Bring your own agent. An agent is a manifest (how to install its CLI in the sandbox, which hosts it talks to, how it logs in) and a small hooks class (how to launch it); everything else is optional. Pass yours as a path (--agent ./my-agent.yaml) or send a pull request; openrua agents lists what is available and what each can do. See docs/agents.md.

Use your own robot

Draft a profile from the robot's live graph, finish the TODO lines, and point up at it:

openrua probe --host > my-ur5.yaml   # joints, limits, frames, ports, cameras from the graph
openrua run ./my-ur5.yaml "..."      # or: openrua config set --robot ./my-ur5.yaml

The profile's machine: section is what the agent's machine.yaml is generated from: model, joint names and limits, frames, gripper, and the ports (trajectory, gripper, twist, wrench) the robot serves. Details in docs/your-own-robot.md.

Simulation and benchmarks

The simulated robots run the community benchmark scenes unchanged; their original success predicates score the trial in place.

openrua bench --config libero_pro --run-id demo \
            --task-suite libero_goal_task --task-ids 0,1 --seeds 0 --operator agent

Every trial writes result.json (verdict, preflight, termination, token accounting), provenance.json (code and simulator commits, image digests, config and prompt hashes), the agent's full transcript, and the workspace it left behind; the run directory keeps a SUMMARY.md regenerated from those files after every trial. Building the simulator checkouts: docs/simulation.md.

A trial replays from its own commands.sh, and a replay with --record renders as a video, terminal on the left, cameras on the right (openrua demo <trial>; see running-experiments.md).

Architecture

Eight units, one direction of dependency: robot/ (the machine, simulated or real), sandbox/ (the agent's terminal and workspace), proxy/ (the only route out), agents/ (the agent contract, registry and launcher), runner/ (bring-up, preflight, operator, verdict, record), demo/ (a recorded trial's files rendered into a video), cli/ and doctor/; under all of them the shared leaves config/ (schema, loader, where things live), errors.py and testing.py. Who may import whom is enforced by CI (import-linter and tests/architecture/). The prose is docs/architecture.md.

Documentation

Results

Claude Code with Claude Opus 5 (effort high), one trial per task and seed, the benchmark's own success predicate, 240 minutes of active wall clock per trial. Details, protocol and per-model runs in docs/demos.md and the paper.

Runs dated 2026-08 to 2026-09.

Citation

The paper will be released soon; until then, cite the software:

@software{openrua,
  author  = {{Terminal World Labs}},
  title   = {OpenRUA},
  year    = {2026},
  url     = {https://github.com/terminalworld/OpenRUA},
  license = {Apache-2.0}
}

License

Apache-2.0