Robo Use

Agent as a policy for embodied agents. Robo Use turns robot benchmarks into tasks that a coding agent, such as Claude Code or Codex, solves by driving a simulated robot from its shell, one robo command at a time. An episode server owns the simulator, enforces the step budget, records a video and scores the final physical state; the agent acts only by sending robo requests to that server.

• 701 tasks in 32 suites ship in the package: rule puzzles on a table, Meta-World, LIBERO, robosuite, RoboCasa, Gymnasium-Robotics, dm_control, MyoSuite, HumanoidBench, quadrupeds, humanoids, drones and more.
• Harnesses for Claude Code, Codex, mini-swe-agent, dimcode and Show-Harness, and a client for vision-language-action (VLA) policies served elsewhere.
• Every task is a BenchFlow task folder (task.md) with a reference solution that scores 1 and a no-op control that scores 0. Every trial is a folder with the video, the agent's trajectory and the reward.

Status: alpha. Interfaces can change between minor versions; see CHANGELOG.md.

Install

Python 3.11 to 3.13, on macOS (Apple silicon) or Linux (tested on arm64):

python3.12 -m venv .venv && . .venv/bin/activate
pip install 'robouse[sim]'

Other simulators are extras, for example robouse[metaworld]; Suites says which task needs which. Models from other providers (--model provider/model) go through a per-trial model gateway, which needs Python 3.12 or newer and the llm extra.

Quickstart (about a minute, no account needed)

Check that MuJoCo can render offscreen. On Linux without a display, install OSMesa first if robouse doctor says it cannot (sudo apt-get install -y libosmesa6).

robouse doctor

Run the task's reference solution, a script that needs no model and no API key:

$ robouse run --task arc-gravity --harness oracle --out runs/quickstart
{
  "trial": "arc-gravity__oracle__8d1bb0",
  "reward": 1.0,
  "outcome": "done",
  "steps": 999,
  "wall_s": 20.0,
  "exception": null
}

In arc-gravity the agent sees example grids of blocks before and after a rule is applied, works out the rule (blocks fall toward the front, like gravity) and moves the real blocks with a gripper. Every task also has a no-op control, which ends the episode without moving and must score 0:

robouse run --task arc-gravity --harness noop --out runs/quickstart

Each run writes a trial folder under runs/quickstart/:

• artifacts/recording.mp4: the video.
• verifier/reward.txt: the reward.
• episode/result.json and episode/trace.jsonl: the verdict and every robo request with its response.

Put an agent in charge

With Claude Code on your PATH and a token from claude setup-token (or an Anthropic API key in ANTHROPIC_API_KEY):

export CLAUDE_CODE_OAUTH_TOKEN=...
robouse run --task arc-gravity --harness claude-code --out runs/quickstart

The agent reads the task, observes the scene with robo observe, moves with robo move-to and ends with robo done. Model trials use your provider quota or credits. mini-swe-agent and dimcode default to GLM-5.3 on the Z.ai Coding Plan (glm/glm-5.3, with GLM_API_KEY and GLM_BASE_URL set). Other harnesses, models and VLA policies: Run agents and VLAs.

Isolation, and its limits

robouse run starts model harnesses under the operating system's sandbox (Seatbelt on macOS, bubblewrap on Linux). The sandbox hides the run folder, the task folder, the installed package, and credential and task-copy locations in your home folder (logins, package caches) from the agent, and the agent's environment carries no credential variables other than the harness's own login or gateway key. But it is a deny list, not a container: the agent runs as your user, can read the rest of your home folder and reach the internet, and public copies of every task, including reference solutions, exist online. Treat it as a guard against accidents, not as a security boundary. Run agents you do not trust in a disposable VM or container, and read SECURITY.md before you publish scores.

Documentation

• Quickstart and Concepts
• Write or compose a task and the task.md reference
• Run a suite or dataset with BenchFlow's bench
• Datasets and tasks and published runs

Community

• Questions and discussion: the BenchFlow Discord.
• Bugs, task issues and feature requests: GitHub issues.
• Contributing: CONTRIBUTING.md. Everyone taking part follows the Code of Conduct.
• Security issues: please report them privately, as SECURITY.md describes, not in a public issue.

Citing

If you use Robo Use in research, please cite it with the metadata in CITATION.cff (GitHub's "Cite this repository" button gives BibTeX), and cite the benchmarks your tasks come from; each suite's page names its source.

License

Robo Use is licensed under the Apache License 2.0. It includes third-party code and task data under their own licenses: MIT (the vendored CaP-X, PyRoKi and RLE-Bench code, LIBERO instructions, DexJoCo demonstrations and others) and CC BY 4.0 (the Adroit demonstrations from D4RL). NOTICE lists every source, its license and the files. Robot models and benchmark assets that Robo Use downloads at run time keep their upstream licenses. Some suites outside the package need non-commercial or NVIDIA-licensed software to run, and the llm extra installs a proprietary LiteLLM component; Suites has the details.