agi-diy · Python

Hosted robot policy inference through LeRobot's remote client. Your application sends joint state and camera frames; predict() returns a chunk of actions. It does not move a robot.

Docs · Browser simulation · Status

First call, no hardware

Use a dedicated Python 3.12 environment (LeRobot, installed on the first inference call, needs 3.12+; agi rollout does not run on 3.14). Sign in at api.agi.diy, create a key and paste it at agi login. In a terminal:

python3.12 -m venv .venv      # or: uv venv -p 3.12
source .venv/bin/activate
python -m pip install agi-diy
agi login
agi run act/so101 --probe -n 1

The first run installs LeRobot's pinned remote client. It then sends black images and a varied synthetic state to the real policy. No motors connect. Today's ACT responds with a (100, 6) float32 chunk; the CLI prints latency, the bill and a trace link. One call costs 100 tokens ($0.01 in usage credit). Without -n 1, the probe makes three calls.

After the dependency check above, save this as probe.py and run python probe.py. This is a second, one-call synthetic test:

import agi
import numpy as np

observation = {
    "observation.state": np.zeros(6, dtype=np.float32),
    "observation.images.top": np.zeros((480, 640, 3), dtype=np.uint8),
    "observation.images.wrist": np.zeros((480, 640, 3), dtype=np.uint8),
}
with agi.Policy("act/so101") as session:
    actions = session.predict(observation)
    print(actions.shape, actions.dtype)
    print("https://api.agi.diy/dashboard/sessions/" + session.id)

For real observations, replace all three arrays with synchronized normalized joint state and correctly placed RGB cameras. act/so101 uses top + wrist; smolvla/so101 uses front + wrist (both 640 × 480 RGB). Query session.features() for the actual negotiated contract. SmolVLA currently returns 50 × 6, not 100 × 6.

Known hardware metadata mismatch: both live checkpoints use normalized arm values −100…100 and gripper 0…100, despite the protocol label so101-degrees-gripper-percent-v1. LeRobot's own so101_follower defaults to use_degrees=true — run your arm with use_degrees=False (--robot.use_degrees=false). Pass it explicitly anyway: 0.4.6+ sets it for you and refuses true; 0.4.5 and earlier do not (pip install -U agi-diy). Verify units before motion. Do not interpret them as degrees or change the handshake identifier to force admission. Validate calibrated transforms, joint order and limits before motor execution. ACT is not language-conditioned; SmolVLA is a sim-trained orange-to-plate policy, not proven arbitrary instruction following. SmolVLA receives task text for action inference; the disabled language-query flag concerns text generation, not this conditioning.

Authentication and sessions

AGI_API_KEY overrides the key saved by agi login in ~/.agi/credentials (mode 0600). AGI_API_URL overrides the API base. Do not log or commit a key.

agi.Policy(model, ttl_s=600) reserves a model process. Connect within 60 seconds. Always use a with block or close() in finally; process-exit cleanup is a fallback, not a substitute for closing. detach() deliberately keeps a lease past process exit.

REST helpers: agi.models(), agi.me(), agi.usage(), agi.sessions(), agi.health().

CLI

Both agi and agi-diy executables are provided. Nothing here moves hardware except agi rollout without --dry-run.

--ttl is a lease length in seconds, 60–3600 (default 600); the API refuses other values.

Hardware: inspect before you run

Calibrate through LeRobot first, check units and camera views, and prepare a clear workspace with a physical stop. This command does not move hardware or create a lease; replace the serial port and camera indices:

agi rollout act/so101 --dry-run -- \
  --robot.type=so101_follower --robot.use_degrees=false --robot.port=/dev/tty.usbmodemYOUR_ARM \
  --robot.cameras='{"top":{"type":"opencv","index_or_path":0,"width":640,"height":480,"fps":30},"wrist":{"type":"opencv","index_or_path":1,"width":640,"height":480,"fps":30}}' \
  --interactive=true --return_to_initial_position=false

Removing --dry-run after validation starts the hardware process. Interactive mode waits for /start before policy playback; connecting hardware may still energize it. Disabling return-to-initial avoids a commanded return motion on exit. Ctrl-C is not a safety-rated stop.

The wrapper installs LeRobot remote/dataset and robot extras, fills the lease flags/FPS, checks camera count and maps names when needed, runs python -m agi._rollout_main in the same interpreter, then ends the lease on exit. Name mapping does not fix camera placement. --fake uses black frames and still state with no hardware; policy calls are still billed. Full rollout guide.

For an existing strands-robots application, session.policy() works with dev/1.0, not the PyPI 0.5.x API. The dev package's Zenoh pin can conflict with LeRobot's; resolve dependencies in a dedicated environment before using it. A bare Policy(session.url) defaults to different encoding/mode and is not equivalent. arm.run(...) executes motion and requires the same safety checks.

Dependencies and failures

The REST layer has no dependencies. Wire inference requires LeRobot remote support, absent from PyPI LeRobot 0.6.1. First use installs lerobot[remote] @ git+https://github.com/huggingface/lerobot@d3793b7e, with GIT_LFS_SKIP_SMUDGE=1, into this interpreter. It uses uv pip when the environment has no pip and repairs transformers outside >=5.4.0,<5.6.0. This can change packages: use an isolated environment. agi.ensure_lerobot() installs ahead of time; AGI_NO_INSTALL=1 disables automatic changes and prints the required command. Python 3.13+ support depends on upstream packages (3.14 was exercised on macOS arm64).

Errors subclass agi.Error and carry .error, .status, .retry_after_s: AuthError, InsufficientCredits, UnknownModel, NoSlots, MissingDependency, RelayDown. Honor the retry delay. predict() reconnects once after a public-relay port change; predict(obs, relay_wait_s=20) can wait for recovery. Do not automatically resume physical motion without checking state after an interruption.

Billing, traces and agents

One returned action = one token = $0.0001. ACT's 100-row call costs $0.01; SmolVLA's 50-row call costs $0.005. Idle leases are not billed; probes and runs of inference again on a recording are. New accounts get 200,000 tokens ($20 in usage credit), once.

Session recording is on by default: state, task, frames and chunks, accessible for 24 hours from session creation. Owners, leasing keys and authorized operators can inspect traces. Policy currently has no record keyword; use REST session creation with record:false (or the JS client) if you need recording disabled. Accounting remains. Viewing traces is not inference; running inference again on a recording is a new billed session on saved inputs, not physical playback.

After agi login, claude mcp add agi-local -- agi mcp connects a coding agent. Local probe spends inference credits; rollout_dry_run does not lease or move. Ask the agent to prepare configuration and seek approval before paid calls or hardware execution. Hosted docs-only MCP needs no install: claude mcp add --transport http agi https://api.agi.diy/mcp.