OpenRUA connects off-the-shelf coding agents to robots through their native ROS 2 interfaces. The agent works in a workspace containing robot documentation and starter tools, writes perception and control programs, and uses execution feedback to continue the task.

You can chat with the agent through OpenRUA's terminal UI or browser, use its original terminal, or run recorded benchmark experiments. Shared chat lets you send follow-up instructions, inspect tool activity, manage the queue, and revisit saved observations without starting a new robot each turn.

• Start a shared session: OpenRUA TUI, browser, or plain CLI.
• Use the agent's original terminal: launch a task with openrua run.
• Connect your robot: describe its ROS 2 interfaces in a profile.
• Run experiments: fresh trials, benchmark scoring, and recorded artifacts.

Quick start

Use a Linux host with Docker or supported Podman setup. The terminal UI is included in the default installation. Automatic session IDs and history selection are available from version 0.3.0:

pip install -U 'openrua>=0.3.0'
openrua

Each ordinary launch creates a new session with an automatic ID; no name is required. The TUI guides you through selecting a robot, simulator, optional benchmark, and coding agent. Names support completion; you can also enter paths to your own profiles. Save & check writes your choices to ~/.openrua/config.yaml and shows the preparation still needed. It does not build images or log in automatically. Follow the commands shown by the check and the installation guide, then choose Save & start.

OpenRUA starts the existing session service in the background and opens chat. Type an instruction and press Ctrl+S, for example:

Inspect the workspace documentation and describe the scene without moving the robot.

Ctrl+Q leaves the interface while the session keeps running. Use /resume in the startup form or chat to search previous conversations by title or ID. You can also run openrua --resume, or openrua --gui --resume ID to open a running conversation and its saved workspace images in the browser. To edit defaults later, use openrua --setup, openrua config set, or edit the same config.yaml; changes apply to new sessions, not the one already running.

Choose End session when finished. Records and workspace files are retained; run openrua again for a new session. Ended or unavailable conversations open read-only from history; selecting them never restarts robot execution. The full walkthrough gives a concrete simulated Panda example with image preparation, queue operations, SSH access, and shutdown.

Shared chat remains experimental. Codex has live shared-session checks; successful Claude Code turns through the shared adapter still need validation. See validation scope.

Choose your interface

The first three use one shared queue per session. These examples explicitly name a session; omitting --name creates a new one with an automatic ID. To reconnect to a running session, pass only its name and interface choice; startup options cannot change an active conversation. Existing chat and session commands remain available for explicit client operations. Native run starts a separate session using the agent's original TUI; it cannot yet take over that shared conversation. For a native terminal with a robot that stays up between agent visits, use up, agent, and down as described in First task in simulation.

In shared chat, new instructions queue behind the active turn. Interrupting pauses the queue for review; Resume continues it. Ending the session stops its resources while preserving files; deleting them is a separate operation. For development, openrua serve still runs the service in the foreground. Stopping that process ends its session, unlike closing one of its clients.

For an optional keyboard-first Pi terminal, install pip install -U 'openrua[pi]>=0.4.0' and run openrua --tui pi. It provides keyboard configuration, /resume, queue management, and tool expansion on the same shared sessions, with no separate Node install. See the Pi guide for controls and current limits.

How it works

OpenRUA TUI / browser / plain CLI
                |
       shared session service
       queue, events, resource ownership
                |
       native coding agent in a workspace
       ROS 2 docs, starter tools, saved files
                |
         ros2 commands / rclpy programs
                |
       robot's native ROS 2 interfaces
       sensors, trajectories, gripper, velocities

The service coordinates user messages and resources. The coding agent decides when to observe, what programs to write, and how to complete the robot task. Native-terminal and benchmark entry points reuse the robot, workspace, and agent adapters without going through the shared chat queue.

• The robot provides the control interface. The agent reads machine.yaml and workspace documentation, acquires sensor data, and sends ROS 2 requests.
• The workspace is the harness. A ROS 2 sandbox contains the agent CLI, documentation, and readable starter-tool source. The agent can inspect, adapt, or replace the tools and save observations and programs for later use.
• Extensions have their own boundaries. Agent manifests and hooks own agent integration; robot profiles own hardware facts; simulator engines and benchmark loaders own their environments. Frontends use the common session API.
• Experiments retain their own protocol. openrua bench creates fresh trials, checks workspace interfaces, and records scoring and provenance separately from interactive conversations.

See Architecture for the module boundaries and contracts.

Choosing what to run

• Robot and scene. --sim selects the simulator; --bench, --task-suite, and --task-id select a benchmark scene. Without a benchmark selection or saved benchmark default, robosuite uses its native Lift scene.
• Agent and model. Choose --agent and optionally --model when starting the session. Switching them inside a running conversation is not implemented.
• Defaults. Save common choices with openrua config set; explicit command arguments override them. Configuration documents the fields.
• Available extensions. openrua robots, openrua simulators, openrua benchmarks, and openrua agents list the bundled and local choices. openrua benchmarks libero_pro lists its suites and tasks.

Supported robots

A robot is what is true of it wherever it runs: joints, limits, frames, gripper, ports, planner. Which simulator embodies it, and what surrounds it, come from the other two kinds of file.

openrua robots prints this list from the files on disk, yours included.

Supported simulators

A simulator file knows the engine and how it drives each robot it embodies; it knows no benchmark. Its install, and the benchmarks' own over it, is what openrua build renders into one image per declaration (openrua-sim-<name>: ROS 2, the checkouts, the assets, the Python environment); docs/simulation.md describes them.

Supported benchmarks

A benchmark names its robot and simulator and brings its own world: install: (its image contents and ROS distro) and scenes: (scene cameras, and robot embodiments its assets add). openrua benchmarks prints this list; openrua bench --config <name> runs one.

Supported agents

Bring your own agent. An agent is a manifest (how to install its CLI in the sandbox, which hosts it talks to, how it logs in) and a small hooks class (how to launch it); everything else is optional. Pass yours as a path (--agent ./my-agent.yaml) or send a pull request; openrua agents lists what is available and what each can do. See docs/agents.md.

Results

Detailed results will be released with the paper.

Use your own robot

Draft a profile from the robot's live graph, finish the TODO lines, and pass it to run:

openrua probe --host > my-ur5.yaml   # joints, limits, frames, ports, cameras from the graph
openrua run ./my-ur5.yaml "..."      # or: openrua config set --robot ./my-ur5.yaml

The profile's machine: section is what the agent's machine.yaml is generated from: model, joint names and limits, frames, gripper, and the ports (trajectory, gripper, twist, wrench) the robot serves. Details in docs/your-own-robot.md.

Simulation and benchmarks

The simulated robots run the community benchmark scenes unchanged; their original success predicates score the trial in place.

openrua bench --config libero_pro --run-id demo \
            --task-suite libero_goal_task --task-ids 0,1 --seeds 0 --operator agent

Every trial writes result.json (verdict, preflight, termination, token accounting), provenance.json (code and simulator commits, image digests, config and prompt hashes), the agent's full transcript, and the workspace it left behind; the run directory keeps a SUMMARY.md regenerated from those files after every trial. Building the simulator checkouts: docs/simulation.md.

A trial replays from its own commands.sh, and a replay with --record renders as a video, terminal on the left, cameras on the right (openrua demo <trial>; see running-experiments.md).

Architecture

The implementation separates robot resources, native agent adapters, shared conversations, and presentation. runner/ composes resources; sessions/ coordinates messages and events; tui/ and web/ provide client interfaces. Robot, sandbox, proxy, and agent knowledge stay in their respective modules. Benchmark execution and recorded demos have separate entry points.

Import-linter and architecture tests enforce these boundaries. See the module map, agent extension guide, and contribution guide.

Documentation

Citation

The paper will be released soon; until then, cite the software:

@software{openrua,
  author  = {{Terminal World Labs}},
  title   = {OpenRUA},
  year    = {2026},
  url     = {https://github.com/terminalworld/OpenRUA},
  license = {Apache-2.0}
}

License

Apache-2.0