get-embodied-tasks

Resolve an embodied-AI task description to its existing dataset and ready-to-run train/eval commands, across pluggable backends. LeRobot is the first backend; more can be added. Ships as both a CLI and an agent skill (Claude + Codex).

get-embodied-tasks "stack blocks"
#  → [lerobot] RoboTwin stack_blocks_two
#    Dataset: lerobot/robotwin_unified
#    Train:   lerobot-train --policy.type=smolvla --dataset.repo_id=lerobot/robotwin_unified ...
#    Eval:    lerobot-eval  --env.type=robotwin --env.task=stack_blocks_two ...

Design

• Facts in the tool, judgment in the skill. The CLI emits real dataset ids and command flags (never model-recalled). The SKILL.md playbook parses a fuzzy task description, calls the CLI, and on a miss researches alternatives or drafts a plan.
• Backends are read, not vendored. The LeRobot adapter reads a vanilla LeRobot checkout by AST over its envs/ source (no import of torch/gymnasium; no patches to LeRobot). RoboTwin/RoboMME task descriptions ship here in data/.
• Drift is surfaced, never hidden. If a backend's expected constant/file changes, the catalog emits structured warnings (stderr, --json warnings, --strict, doctor) so it can be fixed.

Install

pip install -e .
bash install.sh        # symlink into ~/.agents/skills (Claude + Codex pick it up)

Backend discovery

For each backend, in order: $<ENV_VAR> → ./third_party/ei_ws/<name> → ~/ei_ws/<name> → installed package. For LeRobot set LEROBOT_HOME=<checkout> or just pip install lerobot. Configure backends in src/get_embodied_tasks/backends.toml.

CLI

get-embodied-tasks "<query>" [-b BENCHMARK] [--backend NAME] [-n N] [--policy P] [--json]
get-embodied-tasks --benchmark <b>     # list a benchmark
get-embodied-tasks --dump              # whole catalog as JSON
get-embodied-tasks doctor              # backend health / drift report

Reference docs (per backend)

docs/backends/lerobot/ holds reference material for the LeRobot backend, regeneratable from a vanilla checkout:

• cli_reference.md — every lerobot-* command + its extra (generated)
• components_reference.md — registered policies/robots/teleops/cameras/optimizers/schedulers/processors (generated)
• troubleshooting.md — consolidated FAQ (hand-written, links upstream)

Refresh the generated ones:

python docs/scripts/gen_lerobot_reference.py --lerobot-root <checkout>   # or rely on discovery
python docs/scripts/gen_lerobot_reference.py --check                     # CI: fail if stale

Adding a backend

Add an adapter under src/get_embodied_tasks/backends/ implementing backends/base.py:Backend, register it in cli.py:_ADAPTERS, and add a table to backends.toml.