sequence-ai

Cloud inference for robot policies.

pip install sequence-ai

import sequence_ai

with sequence_ai.connect(model="pi05-droid") as policy:
    out = policy.act(
        "pick up the cup",
        observe=robot.read_observation,   # cameras + joints
        act=robot.apply_action,           # one joint target
        validate=robot.is_safe,
        hold=robot.hold,
        max_seconds=30,
    )

policy.interrupt()      # from any thread, at any moment

Deploying your own policy is the same package, on Python 3.10+:

seq init pi05-droid my-pi05          # start from a template: the policy, the robot's client, a README
seq deploy my-pi05/policy.py         # the platform builds and serves it

Two brains, one string between them

A VLM looks at a snapshot, decides what to do next, and says it in a sentence. This package is the how: it takes that sentence with the current cameras and joints, asks a vision-language-action model for the next second of joint targets, and plays them out while fetching the next second.

The two halves run at completely different speeds — the brain thinks between subtasks, the arm needs a chunk every second — and instruction is their entire interface.

act() is shaped to be called as a tool. It overwrites the observation's instruction on every call, so your eyes never have to know what the brain last decided and a stale sentence cannot leak into a chunk after the brain has moved on. max_seconds is required and has no default: this drives hardware, and a tool call a language model can start but nothing bounds is one an arm can be left running by a dropped conversation.

Wiring the two together — retries, failure detection, choosing the next sub-goal — is yours. This package provides inference and a control loop, not orchestration.

Changing your mind

policy.interrupt()

Thread-safe, and the only method meant to be called from a thread other than the one driving.

It stops within one control period, not one chunk. The loop checks between every action, so at 15 Hz the arm stops in about 67 ms. Waiting for the current chunk to finish playing would be up to a full second of an arm still reaching for something the brain has already given up on.

It also cuts through a stalled fetch. If the buffer has run dry and the loop is blocked on a chunk that is taking five seconds, an interrupt does not wait that out — stop latency is a property of this loop and never of the service.

Interruption unwinds through hold() like any other early stop, and is not raised: a stop you asked for is an outcome, not an error. RunOutcome.reason says which ending happened.

What it does for you

Four things, and each of them is something a loop written straight against the HTTP endpoints has to get right before it behaves properly on a robot:

None of the four is guesswork. Every one is a defect that existed in this client, was measured, and was fixed — the numbers below are those measurements, not estimates.

It keeps the connection open

Measured against the production gateway, six samples each:

184.9 ms per call — 60% of the total — is TCP and TLS handshake. That is what with sequence_ai.connect(...) removes. Nothing proprietary: any HTTP client that reuses a connection gets the same result. This package just makes it the default rather than something you have to remember.

Why it matters: a chunk is a deadline. pi05-droid returns 1.00 s of motion per call, and a warm end-to-end /v1/act through this gateway measured 793 ms — the next chunk has to arrive before the current one finishes playing, so 185 ms of avoidable handshake is a fifth of the entire budget.

Actions come in chunks

One call returns a block of future actions, not a single command — between 0.5 s and 2.1 s of motion depending on the model. That is why cloud inference works at all: the control loop does not need a network round trip per control step.

with sequence_ai.connect(model="pi05-droid") as policy:
    pred = policy.predict(observation)
    print(pred.action_chunk)
    # ActionChunk(15 steps x 8 dims, joint_delta, droid_franka, 15 Hz, covers 1.00s, replan after 8)

Read joint_delta in that line before you use the rows. It is the field that decides what the numbers mean, and pi0.5-DROID's are increments — feeding them to a controller that expects absolute joint targets is invisible: same shape, same dtype, values in the normal range, no error, and the arm never completes the task. replan after 8 is the second one: executing all 15 rows when the model asks to be replanned after 8 fails the same quiet way.

You do not have to handle either one. Declare what your controller takes and commands() applies both, plus the gripper exception and the value clip:

policy = sequence_ai.connect(model="pi05-droid", controller="joint_position")
for target in policy.commands(observation):     # already absolute, already 8 rows
    arm.move_to_joint_positions(target)

What to do with the rows covers both controllers and the by-hand route. If you are wiring this into an existing harness rather than starting from run(), read that section first — it is the only part of this document where getting it wrong produces no error at all.

The gap between chunks is the hard part

A loop that waits for the buffer to empty before asking for the next chunk stops the arm once per chunk, every chunk, for a full round trip. It is not an occasional hiccup — it is structural, and on the faster models it dominates:

So run() starts the next inference while the current chunk is still playing, and reports what happened rather than hoping:

out = policy.run(observe=..., act=..., max_actions=250, on_underrun=robot.hold_position)

print(out)        # RunOutcome(250 actions over 17 chunks in 16.8s, completed)
out.underruns     # times the buffer ran dry before the next chunk arrived
out.underrun_s    # total seconds the arm spent with no command
out.max_seam_jump # largest per-dimension step across a chunk boundary

The trade, stated plainly: a prefetched chunk is computed from an observation taken before the previous chunk finished, so the overlap window is open-loop. Freshness and continuity are in direct opposition here and no setting gets both. prefetch=False restores strictly closed-loop behaviour, stall included — right for bench work, wrong for a moving arm.

max_seam_jump is a measurement, not a correction, and it stays one unless you ask otherwise. smooth_seam=N ramps the first N steps of each new chunk out of the last executed action, and it is the only setting in this library that changes a number on its way to the motors — so it is off by default and guarded twice: it applies only when the chunk declares an absolute action space (action_chunk.action_space), and never to the first chunk of a run. Blending deltas is not smoothing; it rescales the increments and moves the arm somewhere the model never asked for, so a delta chunk is passed through untouched even when you ask. max_seam_jump is measured before any blending, so turning it on cannot hide what it smooths.

Many robots at once

One Policy per control loop. A Policy holds one action buffer, and two loops popping from it do not take turns — they interleave. Driving the same handle from a second thread raises, because both quieter options are worse: interleaving sends each robot a shuffled half of the other's plan while both loops report success, and a blocking lock would make the second loop run at half rate, underrunning on every chunk.

def drive(robot, model):
    with sequence_ai.connect(model=model) as policy:      # one each
        return policy.run(observe=robot.read, act=robot.apply,
                          validate=robot.is_safe, max_actions=250)

with ThreadPoolExecutor() as pool:
    left, right = pool.map(drive, [arm_l, arm_r], ["pi05-droid"] * 2)

This costs nothing to follow. The handshake is paid once per Policy, not once per call. Measured with eight loops running concurrently against one gateway: eight connections, ten requests each, zero sequence breaks, zero underruns. Requests arriving while the gateway is busy queue on the server rather than displacing work already in flight.

What the loop says while it runs

run() reports lifecycle, not telemetry:

  starting…      first chunk on its way, or a cold worker loading — nothing is moving yet
  running        a command has reached the robot
  done

Waiting out a cold worker (three to six minutes) and losing a few hundred milliseconds at a chunk boundary are the library's problems, not yours — startup_timeout_s defaults to 1800 s, so the first call on a cold worker waits rather than failing. Every number is still in RunOutcome afterwards if you want it.

policy.run(..., on_status=log.info)     # programmatic
policy.run(..., progress=False)         # silent

The default progress="auto" writes a single line to stderr only when stderr is a terminal — nothing in a script, a pipe, or a log file.

Cold starts

A worker that is not loaded takes three to six minutes to become ready — tens of gigabytes of weights fetched, extracted and JIT-compiled. A control loop's budget for one chunk is 533 ms. Those two numbers are why connecting is a separate call from controlling.

Measured end to end, request to first action:

The worker stays warm for 10 minutes

This is the number that decides how you structure a session. After it becomes ready, a worker is held for 10 minutes past its last request. Every call inside that window is warm — the table's right-hand column. The clock restarts on each request, so a loop that calls even once a minute never goes cold.

Cross that 10-minute gap with no calls and the worker is released, and the next request pays the full cold start again.

call ──► 180-215 s cold start ──► warm ◄──────── 10 min idle window ────────► released
                                    ▲                                            │
                                    └────── any call restarts the 10 min ────────┘

Why ten and not thirty. A held GPU bills whether or not you call it, and the shape of that bill is brutal for a session with thinking time in it: over one real day of use, inference was 3% of the GPU seconds paid for, cold starts 19%, and idle holding 77%. Thirty minutes made the worst case a session that calls once every twenty-five minutes — never cold, almost never working, and billed for all of it. Ten covers a loop and a short pause between runs without paying for a coffee break. {measured 2026-09-16 across 2,147 calls: 322 s of inference against 7,200 s of idle holding}

The practical consequence, if you are running a batch of experiments: the gap between them is what costs you, not the number of them. Ten tasks run back to back pay one cold start. The same ten with a 20-minute analysis gap between each pay ten — that is up to half an hour of waiting that looks like the model being slow and is actually the fleet being released and rebuilt.

wait_until_ready() reports on a worker and starts a cold one, but it does not hold a running one — only acting does. To keep workers warm through the gaps, declare them on the policy (below): warm workers are billed while they run, robot or no robot.

Connect first, then drive

policy.wait_until_ready()          # once, before the robot needs to move
while running:
    policy.next_action(observe())  # every one of these is warm by construction

wait_until_ready() needs no observation — a robot should be able to bring its model up before it is in position, which is exactly when it has no frame worth sending. It is authenticated but not billed: it touches no GPU. It is also what starts the worker, so polling it is the thing that brings the model up, not merely a way to watch.

ready() is the non-blocking form, returning (ready, eta_seconds).

Skipping it does not fail — it silently costs you the loop. Measured on one run against a cold endpoint, the client starting anyway:

first control request   13,533 ms      <- the cold start, now inside the loop
p50                        415 ms      <- the steady state was always fine
mean                     1,096 ms      -> 0.49 arms sustainable   
mean without that one      441 ms      -> 1.21 arms sustainable   

One request that should not have been in the loop is the entire difference between sustainable and not.

If you skip it anyway

The gateway does not stall on a cold worker — it answers 503 immediately with the number:

try:
    policy.predict(observation)
except sequence_ai.Unavailable as exc:
    if exc.warming:
        print(f"loading; ready in ~{exc.retry_after_s}s")   # 174

run() waits that out for you by default (startup_timeout_s=1800), and only before the first action. It reports starting… while it does, so a wait is never mistakable for a hang:

policy.run(..., startup_timeout_s=0)     # opt out: fail immediately on a cold worker

That line is the whole design. Before the first action nothing is moving, so waiting is free. Once the arm is in motion, silently pausing it for two minutes and resuming from a two-minute-old plan is worse than stopping — so mid-run warming is raised, and hold fires.

How big your frames are is how fast you go

Latency scales with the bytes you send, at roughly 16 ms per kB. This is the single largest thing under your control, and it is larger than the model:

observation    p50      mean     sustainable arms
14.6 kB      586 ms    623 ms         0.86        
 3.4 kB      384 ms    433 ms         1.23        

Measured through the gateway, alternating A/B against one warm worker over one connection so that drift in the service cancels. inference_ms did not move between the two (119 ms against 111 ms) — every millisecond of the difference was transport. A direct measurement against the worker agreed: 181 ms for the same 11.2 kB.

Send JPEG, not arrays. A frame as a list of integers is roughly 16x the bytes of the same frame as base64 JPEG, and it is the most common way to land on the slow side of that table.

Use quality 95. Do not go below 90. With the sampling noise pinned, on a real DROID frame:

Quality 85 and below is where the error jumps by an order of magnitude. This section previously read "quality 85 is safe" and quoted the 0.51% figure next to it — but 0.51% is q95's number, and q85 is nearly four times worse. sequence_ai.encode() defaults to 95 so the decision is not yours to get wrong.

Hand it the native frame and let it size it

Do not downscale before calling encode(). Give it whatever your camera produces. It reads the catalogue and does one of two things, and which one is the model's decision, not a setting:

Each model's target is the size its transform produces, so the server's resize is a no-op on arrival — one resample instead of two. Downscaling yourself to some other size is two resamples where one would do, and the intermediate has no claim to being right: training_frame_size is what the dataset stored, not what the model reads.

It runs the server's own operation, so the two agree. For pi05 that is resize_with_pad with the server's filter, bit-identical at every aspect ratio; a frame outside the trained ratio is refused locally with the same message the gateway would 400, rather than cropped or squashed. For GR00T the short-edge resize is shrink-only — a frame already smaller is left alone.

Always JPEG, because that is the format every served handler asks for. Measured in action space against the model's own call-to-call spread, quality 95 is indistinguishable from lossless; the cliff is far below, so there is no reason to spend 3-4x the payload on PNG.

Fidelity below that is not a gentle slope. Degrading the frame further does not degrade the policy proportionally; it stops working. Quantising each channel to four levels — layout, edges, shadows and hues all preserved, a milder change than any sane JPEG setting — took pi0.5-DROID from placing the cube in the bowl in 5.3 s to never lifting it at all, with its closest approach 20x worse than the successful run's final distance. Measured in simulation, 2026-09-11.

Four numbers you should never have to remember

Which camera views a model wants, how wide its state vector is, what aspect ratio it trained on, and what its returned floats mean. All four are per-model, none is guessable, and every one of them fails silently — an unknown view name is not rejected, it arrives as an empty camera; a short state vector is not rejected, it is zero-padded. The robot moves either way; it just moves on something other than what you meant.

So read them off the catalogue instead of remembering them:

model = {m.short_id: m for m in sequence_ai.models(endpoint="act")}["pi05-droid"]

obs = sequence_ai.observation(
    model,
    images={"exterior_1": cam.read(), "wrist_left": wrist.read()},   # ndarray or PIL
    joints=arm.joint_positions(), gripper=[arm.gripper_position()],
    instruction="pick up the cup",
)

That sizes each frame the way the model's own server would — resize_with_pad for pi05, short edge for GR00T — encodes JPEG at quality 95, and raises locally, before anything is billed, if the views, the state width, or the aspect ratio do not match what the model declares. policy.predict() runs the same check and warns when you build an observation yourself; connect(validate="strict") makes it raise, "off" skips it.

state_dim is the input width and is not action_dim. Across the served catalogue it is 8, 17 and 8. Read it off the model, not off this sentence.

state_layout says how that width splits — {"joint_positions": 7, "gripper": 1} for pi0.5-DROID — and the halves are not interchangeable. Eight joints with no gripper sums to 8 and is accepted; the eighth is then dropped and the gripper zero-filled, which holds the hand open for the whole episode. observation() checks each half, not the sum.

What to do with the rows that come back

Tell it what your controller takes, once, and stop deciding.

policy = sequence_ai.connect(model="pi05-droid", controller="joint_position")
for target in policy.commands(observation):
    arm.move_to_joint_positions(target)

That is the whole loop body. commands() applies the model's action space, its conversion reference, its gripper exception and its replan horizon, and hands back rows that are already what your controller expects — the list it returns is what should be executed before observing again, not the whole chunk with a footnote.

controller has two values and no default, because guessing is exactly the failure this removes:

The reference for the increments is taken from the observation you just sent — that is "state_at_request" — so there is no second value to read at the wrong moment.

env = RobotEnv(action_space="joint_velocity", gripper_action_space="position")   # DROID stack
chunk = policy.predict(obs).action_chunk
for i in range(chunk.open_loop_horizon or len(chunk)):
    env.step(sequence_ai.prepare(chunk, model, i))

q_ref = arm.joint_positions()                  # ONCE, before the chunk is requested
for target in sequence_ai.to_joint_positions(chunk, q_ref, model):
    arm.move_to_joint_positions(target)

Three ways to drive it

From most control to least. They are the same request underneath; the difference is who owns the loop.

policy.predict(obs)            # the whole chunk, you do everything
policy.next_action(obs)        # we hold the buffer, you own the cadence
policy.run(observe=, act=)     # we own the loop
policy.act("...", observe=, act=)   # we own the loop and the instruction

act() is run() with the instruction pinned and a time bound required — the shape a VLM calls as a tool. Everything run() accepts, act() accepts.

The other two endpoints: chat and perceive

Act is a loop; these are one-shot calls. Same key, same catalogue, same typed errors.

r = sequence_ai.chat("claude-opus-4-8", [{"role": "user", "content": "what should the arm do?"}])
r.content            # the reply — no choices[0] wrapper
r.tool_calls         # set instead when finish_reason == "tool_calls"

chat() is the slow brain: a sentence in, a sentence (or a tool call) out. It is answered by an external provider with no cold start, so a 503 is an outage rather than a warm-up.

boxes  = sequence_ai.detect("grounding-dino-base", frame, labels=["block", "bowl"])
vecs   = sequence_ai.embed("siglip-so400m", images=[frame], text=["a mug on a plate"])
depth  = sequence_ai.depth("depth-anything-v2-small", frame)   # depth.maps[0] is a numpy array

detect/embed/depth are the three perception tasks. The function you call is the task, so the shape is enforced by the signature: detect requires labels, embed needs images or text or both, depth takes images. A wrong pairing — asking the embedder for depth — is refused locally, before a request is billed, rather than coming back as a 400. Images go through the same encode() the act path uses; a cold perceive worker's warming 503 is waited out.

Perception is not served yet: the platform answers 501 for all three models until it opens. The functions and their local checks are here so code written against them keeps working when it does.

Benchmarks & eval

The other half of the SDK: instead of driving a robot, score a deployed policy on a set of simulation tasks. You write a benchmark the way you write a policy — a decorated class — register it with the same seq deploy, bridge its observations and actions to the policy with an adapter, and run seq eval run. Before spending GPUs, --dry-run shows the estimated wall-clock, the dollar cost, and how many GPUs you will actually be granted. The sims and the policy both run on the platform.

export SEQUENCES_API_KEY=seq_live_...          # identifies your account (owns deployments, billed for evals)
seq init toy-policy                            # toy-policy/toy_policy.py  — a @seq.policy
seq init toy-reach                             # toy-reach/toy_sim.py      — a @seq.benchmark
seq init toy-adapter toy-reach                 # toy-reach/toy_adapter.py  — its adapter, beside it

An adapter runs inside its benchmark's container, so it ships with the benchmark. seq deploy uploads only the entry file, so name the adapter's module on the benchmark's image — in toy_sim.py, image=seq.Image.debian_slim().uv_pip_install("numpy").add_local_python_source("toy_adapter") — or keep the class in the benchmark's own file. Then:

seq deploy toy-policy/toy_policy.py --name toy-reacher    # a @seq.policy deploys
seq deploy toy-reach/toy_sim.py                           # a @seq.benchmark registers (uploads toy_sim.py, toy_adapter.py)
seq eval run toy-reacher --benchmark toy-reach --adapter toy_adapter:ToyReacherAdapter --envs 16 --dry-run
seq eval run toy-reacher --benchmark toy-reach --adapter toy_adapter:ToyReacherAdapter --envs 16

The same from Python — seq.deploy(Cls) does what seq deploy <file> does:

import seq

est = seq.eval("toy-reacher", "toy-reach", adapter="toy_adapter:ToyReacherAdapter", envs=16, dry_run=True)
est.wall_clock_s, est.cost_usd     # an estimate — it starts nothing and reserves nothing

report = seq.eval("toy-reacher", "toy-reach", adapter="toy_adapter:ToyReacherAdapter", envs=16)
report.metrics["success"].overall   # "success" is YOUR metric key from @seq.check — no platform success_rate
report.metrics["success"].per_task  # each task's value with its episode count
report.cost_usd         # actual dollars, reconciled from real GPU-seconds
report.dashboard_url    # the eval in the console
report.videos           # the recordings the platform made (if render= was declared)
report.adapter_lossy    # every lossy transform, with its reason

seq eval report <id> [--wait] [--videos DIR] [--csv PATH] reads an eval later; seq eval ls <policy> lists a deployment's evals over time, and seq eval schedule <policy> --benchmark <name> --every daily runs one on a schedule.

You lay the eval out; the platform never picks the numbers. The policy runs on its own instances (--policy-instances, on the deployment's cards unless --policy-gpu names one) and the benchmark on its own GPU instances (--benchmark-instances, each on an L4 or an RTX-PRO-6000, running --envs environments at once); unset, each is one instance. Every card counts against your account's limit: past it the eval is refused, and while your cards are busy the whole eval waits for them (up to 10 minutes, then it is canceled) — never a silently smaller layout. --dry-run shows the layout, the cards against your account and the estimate. --compare and --sweep add arms to the same eval, run side by side on the same episodes. If the policy and benchmark disagree on obs/action shape and you pass no adapter, eval refuses up front and asks for one rather than handing you a silently-wrong score.

The full reference — the benchmark lifecycle (@seq.setup / @seq.reset / @seq.check / @seq.score, the platform-owned reset→step→check loop), assets= + render=, adapter loss labelling, billing, composite (VLM) evals, and every seq.eval parameter — is in eval.md.

Safety

An action returned by any model is model output, not a safe robot command.

This library does not check joint limits, reachability, collisions, or whether a step is safe at the robot's current velocity. Bounds checking, a watchdog and an e-stop belong between this library and your motors.

policy.run(
    observe=robot.read_observation,
    act=robot.apply_action,
    validate=robot.is_safe,   # return False to stop the loop
    hold=robot.hold,          # called if it stops early, or if act() raises
    max_actions=250,
)

max_actions is required and keyword-only. There is no run_forever() — an unbounded loop that moves a robot should not be startable by accident.

validate=None is allowed for bench and simulation work, and warns once so it cannot happen silently on real hardware.

Declared limits are enforced twice. An action contract can carry bounds (each dimension's [low, high]) and max_delta (the largest change between consecutive steps): seq.ActionContract(..., bounds=[(-0.05, 0.05)] * 6 + [(-1, 1)], max_delta=0.02). The worker clips every chunk into them before it leaves — the response's warnings name each clipped step and dimension — or, with on_violation="refuse", answers 500 bounds_violation instead. The robot client clips again before every action it hands out, measuring max_delta from the previous action even across chunks (a new run() starts afresh). They narrow what reaches the motors; they are not a substitute for the checks above.

Errors are typed

Because a controller reacts differently to each:

Debugging a deployment

When a deployment misbehaves — some control requests slower than expected, or one that just failed — two read-only commands localize the fault and pull the record for a single request by its id:

seq policy doctor <deployment>              # probe the direct-connect chain segment by segment
seq policy logs   <deployment>              # structured logs, connect + worker sinks merged, time-ordered
seq policy logs   <deployment> --rid r-…    # one request: worker phase joined to client timing, attributed

seq policy doctor <name> runs on the platform, segment by segment: the deployment's state and serving version, a running worker's /ping, and one /act on a synthetic observation built from the contract you declared (black frames, zero vectors) — with the round trip split into network vs. worker-handler, so you can tell a slow network from slow inference. It never starts a worker: a deployment scaled to zero says so.

seq policy logs <name> shows your workers' own lines — the model load, every act with its request id and timings, every error with its reason — beside one line per connect and the timings your robots' clients reported. seq policy logs <name> --rid r-… joins one request across all three. Workers built after 2026-10-02 send their lines; an older deployment shows them once deployed again. Lines are kept seven days. Authorization is enforced server-side against your SEQUENCES_API_KEY; you only see your own deployments. For the account side, sequence_ai.account() returns your balance, usage and limits.

Full walkthrough, the segment catalog and the redaction guarantee are in observability.md.

Authoring & deploying your own policy

Everything above drives a model that already runs on the platform. To put your own policy up — declare the environment it needs, point it at your weights, and have the platform validate that environment — use seq: the seq import and the seq command. They come with the same pip install sequence-ai on Python 3.10+; on a 3.9 robot that install stays httpx-only.

A policy is a class decorated with @seq.policy. You declare what it needs — the GPU, the image, where the weights live — and mark its lifecycle methods; nothing is built or downloaded on your machine, the declaration is pure data the gateway builds from.

import seq

@seq.policy(
    gpu="L4",                                  # L4, A100, A100-40GB, H100, RTX-PRO-6000 or "cpu"; a list is
                                               # accepted (every card must be offered), its first card is used
    cpu=4, memory=16,                          # the worker's vCPUs and GB of memory (default 8 / 32)
    image=(
        seq.Image.debian_slim()                # a concrete Debian base, pinned
        .apt_install("git", "libgl1", "libglib2.0-0")   # system libraries
        .run_commands(                         # openpi installs from source: its PyPI name is a placeholder
            "git clone https://github.com/Physical-Intelligence/openpi /opt/openpi",
            "cd /opt/openpi && uv pip install --system -e .",
        )
    ),
    weights=seq.Volume.from_name("pi05-ft"),   # your checkpoint: an uploaded volume, hf://, or a template's (below)
)
class Pi05(seq.Policy):
    @seq.load
    def load(self, weights_dir):               # cold start: pull the checkpoint into VRAM
        self.model = ...

    @seq.infer
    def infer(self, obs, *, seed=None):        # per step: what the robot sends -> what it gets back
        return ...

seq deploy pi05.py uploads that file — and only that file, or its whole package when it lives in one (a directory with __init__.py). A config file or a helper module beside it is shipped when the image names it (below); weights never travel with the code, only through weights=.

The image is declarative

Every seq.Image method returns a new image and builds nothing — the chain is a spec, and the platform builds it into a layer. Start from one base, then extend it; the step order you write is the build order, one for one.

python= picks the worker's Python: "3.11" (the default), "3.12" or "3.13", or "image" for the one the base already has (it must be 3.11 or later). An official python:* base brings its own; a Dockerfile brings its own unless you pass python=.

A build step that needs a credential — a private git repo, a private package index — declares it on that step: .run_commands("pip install git+https://$GH_TOKEN@github.com/acme/private", secrets=[seq.Secret.from_name("gh")]). Only that step sees GH_TOKEN, only while it runs; no image layer and no worker holds it.

The platform's own runtime packages (an HTTP server, pydantic, numpy, pillow) go in right after the base, before your steps — so a version you pin (.uv_pip_install("numpy==1.26.4")) is the one your policy runs with.

@seq.policy(deps=[...]) is shorthand for a trailing .uv_pip_install(...), so a quick policy needs no explicit image= at all. Build-time env carries ordinary variables only — a credential put there is rejected, because the image is not a place for secrets (see below).

Where it runs, how long it may take, and what the build is doing

• region="us-west" (or "eu", "ap", "us-east", … — GET /v1/pricing lists them) starts the workers there, near the robots. A pinned worker is billed at the region's multiple of the card's price (1.15× for a broad region like "eu", 1.75× for a narrow one like "us-west"); seq policy estimate <file> shows it.
• timeouts=seq.Timeouts(startup=900, request=5): a worker that has not loaded within startup seconds fails (the build check is held to the same), and one inference longer than request seconds is answered 504 while the worker finishes it before serving again. Defaults: 1800 and 300.
• seq deploy shows each phase of the build as it begins — the image, the weights, the worker loading, the /act check — with the newest line of the build's log; seq policy logs <name> --build prints the newest version's phases and its whole log, then or later.
• seq secret set <name> reaches deployments that declare the secret without a re-deploy: their next workers start with the new value (running ones keep the old one until they are replaced).

Before it serves: one real /act

A policy that declares observation= is proven before its first robot: the build ends by calling its /act once on a synthetic observation — black frames at each declared camera size, zero vectors of the declared lengths, an instruction. A policy that cannot answer it fails the build, and seq deploy shows the policy's own error instead of a robot finding it later. Without an observation contract there is nothing valid to send, so only the load is checked.

Knobs a request may turn: inference parameters

A keyword of @seq.infer with a default is an inference parameter: def infer(self, obs, *, seed=None, replan_steps: int = 5) declares replan_steps. Its type is the annotation (or the default's): int, float, bool or str. seq validate lists them; a robot sets one per call with policy.predict(obs, params={"replan_steps": 3}), and an eval with seq eval run … --param replan_steps=3 (or --sweep replan_steps=2,5 to compare values). The worker casts each to its declared type and refuses, before any inference, a name the policy does not declare (400 params). Unset, the default applies.

State across steps, and more than one robot at a time

• streaming=True (with an @seq.on_episode method) keeps every /act of one episode on the same replica, so state your class holds — a KV cache, a frame window — carries over. policy.run() ends the episode when it returns, even on an exception, and @seq.on_episode frees that state; an episode left idle for 5 minutes (a robot that crashed) is ended by the worker. A policy that only needs the last N frames needs no state: declare observation=seq.ObservationContract(history=N) and the robot sends them. (frames_per_chunk= is retired: nothing on the platform ever read it.)
• @seq.concurrent(max_inputs=N, target_inputs=N) under @seq.policy lets one worker hold N requests at once; without it a second robot acting at the same moment waits behind the first (429, retried by the client). target_inputs is also how many robots one worker takes before another starts (next point).
• More workers, warm workers. @seq.policy(max_containers=3) lets a deployment run up to three workers: each connected robot joins the least-busy worker with room (target_inputs robots each), another starts when all are full, and a robot keeps its worker across its lease renewals. Undeclared, a deployment runs one worker that every robot shares. min_containers=1 keeps that many running even with no robot connected — no cold start, billed while they run. scaledown_window=300 is how long an idle worker stays (default 600 s).

Weights live on a volume, never in the image

weights= is copied once, at deploy, into a volume the platform mounts at /data — never baked into an image layer (so a multi-GB checkpoint never bloats the build), and every cold start reads that copy. It names one of three things:

• An official template's weights — seq.Weights.template("pi05-droid", version=1) (the string form is "template://pi05-droid@v1"): the unmodified checkpoint a template ships with, plus what it loads at runtime (pi0.5's tokenizer). A published version never changes; leave version= out to take the newest one at each deploy. No key is needed. This is what seq init pi05-droid writes.

• Hugging Face — hf://org/model or hf://org/model@<revision>. At deploy the revision is resolved to a commit and exactly that commit is copied, so a re-deploy is deterministic. A private or gated repo needs a token: store it once (seq secret set hf --env HF_TOKEN) and declare it, secrets=[seq.Secret.from_name("hf")]. The copy runs with only that token; declaring it also hands HF_TOKEN to your policy at runtime. A repo the platform cannot read — private without a token, gated, a wrong revision — is refused when you deploy, in seconds, with what to fix.

• Your own files — upload a local checkpoint directory once, then deploy it by name:

  seq volume put pi05-ft ./checkpoints/pi05-ft     # resumable: run it again after an interruption
  seq volume ls                                    # name, state, version, size
  

  @seq.policy(weights=seq.Volume.from_name("pi05-ft"), ...)    # or weights="vol://pi05-ft@v2" to pin
  

  Every file's md5 is checked when the upload completes; only then does the version become active (and billed as storage). A new put of the same name makes the next version. seq volume rm is refused while a deployment's weights, or a registered benchmark's assets, use the volume. Nothing is lost at once: a deleted volume, and the version a new put replaced, wait 7 days in the recycle bin, not billed.

  seq volume ls --trash                            # what is in the bin, and until when it can come back
  seq volume restore pi05-ft                       # bring back the latest deleted or replaced version
  seq volume restore pi05-ft --version 2 --as pi05-ft-v2   # a given version, under another name
  seq volume rm pi05-ft --purge                    # delete now, skipping the bin
  

gs://, s3:// and https:// weights are deprecated. This release still deploys them, with a warning; the next one refuses them. Move the files into a volume once — download them, then seq volume put <name> <dir> — and deploy weights=seq.Volume.from_name("<name>"), or point at the Hugging Face repo they came from.

Weights are validated as you write them: an unknown scheme, a .. traversal, an hf:// without a repo, or a template name that is not one is rejected with a message that names the problem.

seq.Secret.from_name(name) references a credential by name; the plaintext never enters the SDK or the deploy payload. Explicit volumes= (seq.Volume.from_name(...)) is not supported yet: a deploy that declares one is refused rather than run without it.

Versions, rollback, stop

Deploying a name that already serves never takes it down: the new version builds beside the serving one, robots keep acting on the old version the whole time, and the new one takes over only once its build is ready. A version that fails to build changes nothing — the old one goes on serving, and seq deploy says so. New sessions go to the new version at once; a robot already acting moves to it at its next lease renewal (a few minutes), onto a worker started for it in advance.

seq policy ls                        # your deployments: the serving version, one building beside it, the card
seq policy versions pi05-ft          # v4 building · v3 serving · v2, v1 kept
seq policy rollback pi05-ft          # serve the previous version again — instantly, nothing is rebuilt
seq policy rollback pi05-ft --to 1   # or a specific kept version
seq policy stop pi05-ft              # emergency stop: every worker ends now, every connect is refused
seq policy start pi05-ft             # let robots connect again
seq policy rm pi05-ft                # delete it: stops its workers now, frees the name and the quota slot

The serving version and four earlier ready ones are kept; a sixth pushes the oldest out.

A canary. seq policy traffic <name> v5=10 v4=90 sends one new session in ten to v5 and the rest to v4 (any versions seq policy versions lists as serving or kept); seq policy traffic <name> shows the split, --clear sends everything to the serving version again. A session keeps the version it started on for its lease renewals, so a robot never switches versions mid-run. To try a version on one robot first, pin that robot's key (a key bound to this deployment) to it: seq policy traffic <name> --pin arm-7 v5; --unpin arm-7 lets it follow the split again. A pin reaches every gateway within five minutes.

Budget, schedule, alerts, and the price before you deploy

• seq policy estimate policy.py [--hours-per-day 8] prices the policy's card from the platform's list: its warm floor around the clock, or the hours a scale-to-zero deployment is in use. Nothing is built.
• seq policy budget <name> --monthly 100: when the month's spend reaches it, the deployment stops — workers end, connects are refused and say why. To start it again, raise the budget (or --off), then seq policy start <name>; under a budget already reached, seq policy start is refused.
• seq policy scale <name> --min 2 --during "Mon-Fri 08:00-18:00" --tz America/Los_Angeles keeps two workers warm in those hours (billed whether or not a robot is connected); --clear removes it.
• seq policy alerts <name> --webhook https://… [--error-rate 0.05] [--p95-ms 500] [--budget-pct 80] posts JSON to your https endpoint when a version fails to build, the hour's error rate or p95 crosses the threshold, or the budget nears or is reached. Each body is signed: X-Seq-Signature: sha256=HMAC(secret, body) with the secret printed once.

Field data: capture, label, export

seq policy capture <name> --rate 0.1 [--images] makes the deployment's workers keep a tenth of its episodes (each kept whole: the observation it sent, the chunk it got, the inference parameters), with the camera frames when you ask. No redeploy: running workers pick it up within a minute. A robot marks how an episode went with policy.label(success=True, notes="") after its run(). seq policy data export <name> --format lerobot --out DIR writes a LeRobot v2.1 dataset (one frame per /act, the chunk's first action, the episode's label as next.success; needs pip install 'sequence-ai[data]'), or --format jsonl the files as kept. Kept data is billed as the capture:<name> dataset volume, removed after --retention-days (default 30), and deleted with the deployment. seq policy capture <name> --off stops it.

Have the platform validate it

seq init pi05-droid my-pi05          # scaffold a working template to edit
seq validate my-pi05/policy.py       # or a class in a file:  seq validate my_policy.py:Pi05

seq validate reads the policy's declaration — it never runs load or infer — into the deployment request, validates it against the platform's contract, and echoes back exactly what will be installed, where the weights mount and which files are uploaded — with nothing built, run or sent:

policy      my-pi05
platform    accepted  (validated against deployment contract 0.4.14)
image       nvidia/cuda:12.2.2-cudnn8-runtime-ubuntu22.04
  apt-get install   git, git-lfs, build-essential, clang, linux-headers-generic, libgl1, libglib2.0-0
  run               pip install --no-cache-dir uv
  run               git clone --recurse-submodules https://github.com/Physical-Intelligence/openpi /opt/openpi
  run               cd /opt/openpi && GIT_LFS_SKIP_SMUDGE=1 uv pip install --system -e .
  env               OPENPI_DATA_HOME=/data/.cache/openpi
weights     template://pi05-droid@v1
            → mounted at /data  (provisioned by the platform onto a volume, not into the image)
code        1 file(s), 4.8 KiB uploaded:
              policy.py
action      droid_franka · joint_velocity · 8 dims · 15 Hz  (every chunk is checked against this)
observation exterior_1 224x224, wrist_left 224x224 · joint_positions[7] · gripper[1] · instruction  (every /act is checked against this)

Only policy.py goes up: the robot's client.py, the README and the tests beside it stay on your machine.

When it reports accepted, seq deploy my-pi05/policy.py hands the build to the platform (it needs SEQUENCES_API_KEY); seq deploy … --dry-run prints the same without sending it. A malformed declaration fails here, at authoring time, with a readable error and a non-zero exit — not as an opaque build failure minutes later.

Configuration

export SEQUENCES_API_KEY=seq_live_...      # or pass api_key= to connect()

Get a key at app.generalsequences.com.

Secrets — bring your own keys (BYOK)

Some policies need a credential you own: a Hugging Face token for private hf:// weights, or your own OpenRouter key for the VLM in a composite policy (we run the low-level policy on GPU and bill for that; the VLM calls go to your OpenRouter account and never touch us). You store those keys once, by name, and refer to them from a policy by that name alone — the key value never goes into your code, your logs, or any output.

A secret is one name bound to a set of environment variables. The two canonical bundles are expressible by name alone:

seq secret set hf         --env HF_TOKEN            # private hf:// weights (and your policy's own use)
seq secret set openrouter --env OPENROUTER_API_KEY  # your VLM key for a composite policy

Values are read from stdin, never the command line

There is no --env KEY=VALUE and no --value flag — a value never lands in argv, your shell history, or the process listing. Pipe it in, or type it at the hidden prompt:

printf '%s' "$OPENROUTER_API_KEY" | seq secret set openrouter --env OPENROUTER_API_KEY
# or, interactively, you are prompted per key with the input hidden:
seq secret set openrouter --env OPENROUTER_API_KEY
#   Value for OPENROUTER_API_KEY (input hidden):

A bundle with several env vars reads one line per --env, in order:

printf '%s\n%s\n' "$WANDB_API_KEY" "$WANDB_ENTITY" | seq secret set wandb --env WANDB_API_KEY --env WANDB_ENTITY

List and remove — names only, never values

seq secret ls          # name + env-var NAMES + timestamps; the value is never shown, ever
seq secret rm openrouter

ls is metadata only. There is no command that reads a stored value back — from the SDK a secret is write-only. Re-running seq secret set <name> rotates the key (it replaces the whole bundle); no delete first. A rotated value goes live on the next worker start; seq policy stop <name> then seq policy start <name> replaces the running workers at once.

Use it from a policy — by name only

import seq

@seq.policy(gpu="L4", secrets=[seq.Secret.from_name("openrouter")])
class MyPolicy(seq.Policy):
    ...

Secret.from_name carries the name and nothing else — no value parameter exists, and it does no network I/O. seq deploy checks at deploy time that the name exists and is yours.

set/ls/rm each require your key (export SEQUENCES_API_KEY=seq_live_...) and are scoped to it: another key never sees your secrets. A missing or invalid key fails closed — nothing is stored, listed, or deleted.

One install, two halves

pip install sequence-ai on Python 3.10+ brings everything: the robot client (sequence_ai), the seq command and the decorators, with pydantic and the shared contract package sequence-base. On Python 3.9 — a robot's interpreter — the same install is the client alone, and seq says it needs 3.10+ instead of a stack trace.

Limits

What the platform fixes; everything else is your code's. Each value is what the platform runs today.

Install footprint

On a robot (Python 3.9), one dependency: httpx. On Python 3.10+, seq adds pydantic and sequence-base.

A robot controller is often on a Jetson with a pinned, fragile Python environment, and ROS 2 Humble ships Python 3.10. Every transitive dependency is another chance for the install to fail on the machine that actually matters.