sidekick-sdk

Python client for the Sidekick Robotics API: one endpoint for every robotics foundation model, with routing, fallback and per-request billing.

pip install sidekick-sdk

One dependency (httpx). It installs on a robot control computer, or inside a ROS container, without dragging a web framework along.

Get an API key at sidekickrobotics.ai/api.

Sixty seconds

from sidekick_sdk import Sidekick

sk = Sidekick(api_key="sk-sidekick-...")

act = sk.act(
    model="sidekick/auto:manipulation",   # ask for a job, not a checkpoint
    observations=[sk.observation(image_path="frame.jpg", proprio=joints)],
    instruction="put the spoon on the towel",
    action_space="joint_pos_14",
    horizon=16,
)

print(act.route["model"])      # which checkpoint actually served you
print(act.usage["cost_usd"])   # what it cost
robot.execute(act.steps)

model takes a router alias (sidekick/auto:manipulation), a concrete model id, or a list via models=[...] in the order you want them tried. Every response carries route, so you always know what answered and whether it fell back.

Say what you care about

sk.act(..., preference="fastest")   # balanced | fastest | cheapest | reliable

fastest orders live candidates by measured latency, cheapest by price, reliable by success rate, balanced blends them. Add hard constraints alongside it:

sk.act(..., provider={"max_latency_ms": 900, "allow_simulated": False})

allow_simulated=False is the one to set for anything with an actuator behind it. Without it a placeholder response is structurally indistinguishable from a real one.

Four contracts, on purpose

They are not interchangeable, and the router will not silently serve one where you asked for another.

Control loops

The one thing that catches people: you cannot call a cloud API once per actuation. A control loop runs at 30 to 200 Hz and a policy answers in several hundred milliseconds. /v1/act therefore returns a chunk of N future actions plus the dt_ms they were planned for; the robot executes that chunk from a local buffer while the next one is fetched in the background.

ActionStream is that buffer, and the timing it gets right is not obvious:

stream = sk.stream_actions(
    model="sidekick/auto:manipulation",
    instruction="fold the towel",
    action_space="joint_pos_14", dof=14,
    preference="fastest",
    observe=lambda: sk.observation(cameras=rig.capture(), proprio=robot.joints()),
)

stream.warm_up()          # cold container, arm still braked
with stream:              # starts the background policy thread
    while running:
        step = stream.next_action()
        if step is None:
            robot.hold()  # policy fell behind; decelerate, never repeat
        else:
            robot.set_joint_positions(step)
        time.sleep(stream.dt_s)

Three things worth knowing about it:

• next_action() never blocks and never raises. All network work happens on the policy thread. Your servo loop stays real-time.
• None is a real answer. The buffer is dry or the chunk is stale. Hold or decelerate. Repeating the last command indefinitely is a robot acting on a world that has moved on.
• Refills are triggered in time, not in steps. "Refetch when 5 steps are left" sounds right and stalls: 5 steps at 66 ms is 330 ms of runway against a 684 ms fetch. The stream measures its own round trip and refills at 1.5x it.

stream.check() runs three guards on every chunk before it can reach an actuator, raising UnsafeResponse on a simulated route, a mismatched action space, or a step whose width is not this robot's DOF.

Multi-camera rigs

obs = sk.observation(
    cameras={"cam_high": "frames/high.jpg",
             "cam_low": "frames/low.jpg",
             "cam_left_wrist": "frames/lw.jpg",
             "cam_right_wrist": "frames/rw.jpg"},
    proprio=joint_positions)

Values can be URLs, local paths or base64. Camera names must match what the checkpoint declares: a wrist view passed as the head view produces confident nonsense, and nothing in the response says so.

Discovery

sk.routes()                       # router aliases that are live right now
sk.models(domain="manipulation")  # the catalog
sk.taxonomy()                     # every value the filters accept, with counts
sk.action_space_dims("joint_pos_14")
sk.preview_route(model="sidekick/auto:manipulation", contract="act")  # free dry run
sk.usage()                        # your ledger

Call taxonomy() instead of hard-coding domain, family or action-space strings.

Configuration

Sidekick(api_key=..., base_url=..., timeout=120.0, max_retries=2)

base_url defaults to $SIDEKICK_ORIGIN, then to the public gateway. Point it at a private deployment without touching call sites.

Errors

SidekickError carries .status, .code and .body. UnsafeResponse is separate on purpose: a SidekickError means the call did not succeed and you should retry; an UnsafeResponse means the call succeeded and the answer is wrong for this robot, which is the more dangerous case because nothing about it looks broken.

Full API reference: https://www.sidekickrobotics.ai/api

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md.

License

Apache-2.0. See LICENSE.