sequence-ai

Deploy a robot policy, then drive a robot with it.

pip install sequence-ai                  # Python 3.10+ for `seq`; a 3.9 robot gets the client alone
export SEQUENCES_API_KEY=seq_live_...    # create one at app.generalsequences.com
seq init pi05-droid my-pi05              # an official template: the policy, the robot's client.py, a README
seq deploy my-pi05                       # the platform builds it and serves it, as your own deployment

A key starting seq_live_ is a full access key: it acts for your account, so the CLI and your own code use it. A key starting seq_robot_ is a robot key: made on a policy's Robot keys tab in the console, it drives that one policy and nothing else.

What changed in each release: generalsequences.com/docs/changelog.

On macOS, /usr/bin/seq (a system tool that prints numbers) may come first on your PATH: check that which seq points to your Python environment's bin, or run the CLI as python -m seq ….

import sequence_ai

with sequence_ai.connect(model="my-pi05") as policy:    # your deployment, by the name you gave it
    out = policy.run(
        observe=robot.read_observation,   # cameras + joints
        step=robot.apply_action,          # one row of the chunk, as your policy defines it
        instruction="pick up the cup",
        validate=robot.is_safe,
        hold=robot.hold,
        max_seconds=30,
    )

policy.interrupt()      # from any thread, at any moment

run() is the loop, ready-made. Taken apart, it is a sequence_ai.Player — for a loop of your own — and policy.act(observation), one request (Three layers).

There is no shared model to call: every policy runs as a deployment of the account that deployed it, started from a template (seq init --list) or from your own code. seq init <template> writes the template's stable version when this sequence-ai can deploy it, and otherwise the newest version it can, saying which and why; @stable, @beta or @vN after the name is written as named, with a warning when this sequence-ai cannot deploy it.

Two brains, one string between them

A VLM looks at a snapshot, decides what to do next, and says it in a sentence. This package is the how: it takes that sentence with the current cameras and joints, asks your deployed policy for the next second of actions, and plays them out while fetching the next second.

The two halves run at completely different speeds — the brain thinks between subtasks, the arm needs a chunk every second — and instruction is their entire interface.

run(instruction=...) is shaped to be called as a tool. It overwrites the observation's instruction on every act of the run, so your eyes never have to know what the brain last decided and a stale sentence cannot leak into a chunk after the brain has moved on. A run needs a bound — max_seconds=, max_actions= or both — and has no default: this drives hardware, and a tool call a language model can start but nothing bounds is one an arm can be left running by a dropped conversation.

Wiring the two together — retries, failure detection, choosing the next sub-goal — is yours. This package provides inference and a control loop, not orchestration.

Changing your mind

policy.interrupt()

Thread-safe, and the only method meant to be called from a thread other than the one driving.

It stops within one control period, not one chunk. The loop checks between every action, so at 15 Hz the arm stops in about 67 ms. Waiting for the current chunk to finish playing would be up to a full second of an arm still reaching for something the brain has already given up on.

It also cuts through a stalled fetch. If the buffer has run dry and the loop is blocked on a chunk that is taking five seconds, an interrupt does not wait that out — stop latency is a property of this loop and never of the service.

Interruption unwinds through hold() like any other early stop, and is not raised: a stop you asked for is an outcome, not an error. RunOutcome.reason says which ending happened.

What it does for you

Four things, and each of them is something a loop written straight against the HTTP endpoints has to get right before it behaves properly on a robot:

None of the four is guesswork. Every one is a defect that existed in this client, was measured, and was fixed — the numbers below are those measurements, not estimates.

It keeps the connection open

Measured against the platform, six samples each:

184.9 ms per call — 60% of the total — is TCP and TLS handshake. That is what with sequence_ai.connect(...) removes. Nothing proprietary: any HTTP client that reuses a connection gets the same result. This package just makes it the default rather than something you have to remember.

Why it matters: a chunk is a deadline. A pi0.5-DROID chunk is 1.00 s of motion, and the next one has to arrive before it finishes playing, so 185 ms of avoidable handshake is a fifth of the entire budget.

On Linux, a connection that sent nothing for longer than its retransmission timeout restarts its congestion window (net.ipv4.tcp_slow_start_after_idle, on by default), so an act carrying camera frames can take one more round trip to upload. The SDK has its connection to the worker use BBR where the robot allows it (BBR does not restart; sudo modprobe tcp_bbr makes it available, and the SDK then takes it on its own), and otherwise keeps that connection busy between acts with a small request to the worker. That request cannot go while an act waits for its answer, so for a policy whose inference takes longer than about 0.2 s (pi0.5 on an L4 takes about that) the window still restarts during the wait and about half of the acts keep the extra round trip; for such a policy load BBR, or turn the restart off on the robot: sudo sysctl -w net.ipv4.tcp_slow_start_after_idle=0 (in a container, which has its own copy of it: docker run --sysctl net.ipv4.tcp_slow_start_after_idle=0 ...).

Actions come in chunks

One call returns a block of future actions, not a single command — as many steps as your policy returns, at the rate it declares. That is why cloud inference works at all: the control loop does not need a network round trip per control step.

with sequence_ai.connect(model="my-pi05") as policy:
    answer = policy.act(observation)
    print(answer.action_chunk)
    # ActionChunk(15 steps x 8 dims, joint_velocity, droid_franka, 15 Hz, covers 1.00s, replan after 8)

The rows are your policy's own output, untouched. Nothing converts them — not the platform, not this client — so what they mean is what the policy declares, and joint_velocity in that line is the one to read: pi0.5-DROID returns seven joint velocities and a gripper position, and feeding them to a controller that expects joint positions is invisible — same shape, same dtype, values in a normal range, no error, and the arm never completes the task. policy.to_command(step) turns a row into the command your controller runs, by what the deployment declares (below). replan after 8 is the other field: executing all 15 rows when the policy asks to be replanned after 8 fails the same quiet way. answer.rows is the 8 to execute and answer.spare the other 7, which run() and a Player play only while the next chunk is late.

The gap between chunks is the hard part

A loop that waits for the buffer to empty before asking for the next chunk stops the arm once per chunk, every chunk, for a full round trip. It is not an occasional hiccup — it is structural, and on short chunks it dominates:

So run() starts the next inference while the current chunk is still playing, and reports what happened rather than hoping:

out = policy.run(observe=..., step=..., max_actions=250, on_underrun=robot.hold_position)

print(out)            # RunOutcome(250 actions over 17 chunks in 16.8s, completed)
out.underruns         # times the buffer ran dry before the next chunk arrived
out.underrun_s        # total seconds the arm spent with no command
out.spare_steps_used  # steps past a chunk's open_loop_horizon, played while the next chunk was late
out.max_seam_jump     # largest per-dimension step across a chunk boundary
out.max_plan_age_s    # the oldest plan a step was played from: seconds since its act was sent

Each run's counts also go to the platform, where seq usage --policy <name> and the policy page show how many chunks were on time and the stalls: when the run ends, or - if the process ends in the middle of it (SIGTERM from its process manager, SIGHUP, its exit) or the Policy is closed - as it stood then. A run ended by SIGKILL (kill -9, an out-of-memory kill) ends before anything can be sent, and is not reported. A Player's episode is reported the same way, when its with block ends. With the counts go the oldest plan a step was played from and, for absolute steps, the largest seam jump: the policy page's tip on the stalls names max_age_s when a plan 5 s old or older was played, and smooth_seam when a seam jumped.

The trade, stated plainly: a prefetched chunk is computed from an observation taken before the previous chunk finished, so the overlap window is open-loop. Freshness and continuity are in direct opposition here and no setting gets both. prefetch=False restores strictly closed-loop behaviour, stall included — right for bench work, wrong for a moving arm.

max_seam_jump is a measurement, not a correction, and it stays one unless you ask otherwise. smooth_seam=N ramps the first N steps of each new chunk out of the last executed action, and it is the only setting in this library that changes a number on its way to the motors — so it is off by default and guarded twice: it applies only when the chunk declares an absolute action space (action_chunk.action_space), and never to the first chunk of a run. Blending deltas is not smoothing; it rescales the increments and moves the arm somewhere the model never asked for, so a delta chunk is passed through untouched even when you ask. max_seam_jump is measured before any blending, so turning it on cannot hide what it smooths.

Many robots at once

One Policy per control loop. A Policy plays one action buffer at a time — its run()'s or its one Player's — and two loops popping from it do not take turns — they interleave. Driving the same handle from a second thread raises, and so does a second Player on it, because both quieter options are worse: interleaving sends each robot a shuffled half of the other's plan while both loops report success, and a blocking lock would make the second loop run at half rate, underrunning on every chunk.

def drive(robot, model):
    with sequence_ai.connect(model=model) as policy:      # one each
        return policy.run(observe=robot.read, step=robot.apply,
                          validate=robot.is_safe, max_actions=250)

with ThreadPoolExecutor() as pool:
    left, right = pool.map(drive, [arm_l, arm_r], ["my-pi05"] * 2)

This costs nothing to follow: the handshake is paid once per Policy, not once per call. How many robots one deployment serves at once is its own declaration — each worker's seats (one robot a model copy, unless its infer declares @seq.concurrent(max_concurrent=N)) and as many workers as its GPUs allow (max_gpus=, your account's limit; below); a robot that finds every slot taken and no GPU left waits in line before its loop starts, or is refused with on_full="reject".

What the loop says while it runs

run() reports lifecycle, not telemetry:

  starting…      first chunk on its way, or a cold worker loading — nothing is moving yet
  running        a command has reached the robot
  done

Waiting out a cold worker (as long as your policy takes to load) and losing a few hundred milliseconds at a chunk boundary are the library's problems, not yours — startup_timeout_s defaults to 1800 s, so the first call on a cold worker waits rather than failing. Every number is still in RunOutcome afterwards if you want it.

policy.run(..., on_status=log.info)     # programmatic
policy.run(..., progress=False)         # silent

The default progress="auto" writes a single line to stderr only when stderr is a terminal — nothing in a script, a pipe, or a log file.

Cold starts

A worker that is not running starts when a robot connects: the platform starts a machine with your image, mounts your weights and runs your @seq.load (and @seq.warmup). For a multi-gigabyte checkpoint that is minutes — the weights read, the model compiled — while a control loop's budget for one chunk is a few hundred milliseconds. Those two numbers are why connecting is a separate call from controlling.

A worker stays warm for idle_timeout

After its last request a worker stays for idle_timeout seconds — 600 unless your policy declares otherwise — and every call inside that window is warm. The clock restarts on each request, so a loop that calls even once a minute never goes cold. Cross the window with no calls and the worker stops; the next connect pays the cold start again.

connect ──► cold start ──► warm ◄────────── idle_timeout idle ──────────► stopped
                             ▲                                              │
                             └────── any call restarts the window ──────────┘

Why 600 by default, and not longer. A held GPU bills whether or not you call it, and a session with thinking time in it is mostly holding: ten minutes covers a loop and a short pause between runs without paying for a coffee break. The trade is yours: idle_timeout=1800 on @seq.policy holds longer, min_workers=1 keeps a worker running with no robot connected (no cold start, billed while it runs), and seq policy scale keeps some warm in working hours only.

The practical consequence, if you are running a batch of experiments: the gap between them is what costs you, not the number of them. Ten tasks run back to back pay one cold start; the same ten with a 20-minute analysis gap between each pay ten.

wait_until_ready() reports on a worker and starts a cold one, but it does not hold a running one — only acting does.

Connect first, then drive

policy.wait_until_ready()                  # once, before the robot needs to move
with sequence_ai.Player(policy) as player:
    while running:
        robot.step(player.next(observe))   # every act behind these is warm by construction

wait_until_ready() needs no observation — a robot should be able to bring its model up before it is in position, which is exactly when it has no frame worth sending. It is authenticated but not billed: it touches no GPU. It is also what starts the worker, so polling it is the thing that brings the model up, not merely a way to watch.

ready() is the non-blocking form, returning (ready, eta_seconds) - a Ready that, once a worker serves, also says the card it runs on: card and usd_per_hour (one worker's price an hour on it, as billed).

Skipping it does not fail — it silently costs you the loop. Measured on one run against a cold endpoint, the client starting anyway:

first control request   13,533 ms      <- the cold start, now inside the loop
p50                        415 ms      <- the steady state was always fine
mean                     1,096 ms      -> 0.49 arms sustainable   
mean without that one      441 ms      -> 1.21 arms sustainable   

One request that should not have been in the loop is the entire difference between sustainable and not.

If you skip it anyway

Connecting to a cold deployment does not stall — it answers 503 at once with the number, which is what you see with startup_timeout_s=0:

try:
    policy.act(observation)              # connect(..., startup_timeout_s=0)
except sequence_ai.Unavailable as exc:
    if exc.warming:
        print(f"loading; ready in ~{exc.retry_after_s}s")   # 174

By default the SDK waits that out for you (startup_timeout_s=1800): act() on every call, run() and a Player only before the first action. It reports starting… while it does, so a wait is never mistakable for a hang:

policy.run(..., startup_timeout_s=0)     # opt out: fail immediately on a cold worker

That line is the whole design. Before the first action nothing is moving, so waiting is free. Once the arm is in motion, silently pausing it for two minutes and resuming from a two-minute-old plan is worse than stopping — so mid-run warming is raised: run() fires hold, and a Player's next() raises it into your loop.

A deployment whose every slot is taken, with no GPU left to start another worker, is waited out the same way: the robot is put in line before its control loop starts, and connect() asks again at the pace your key's rate limit allows, printing [sequences] queued for a GPU slot: position 2, about 30s … (on_queue= replaces the line). It gives up after the deployment's queue_timeout_s with the reason in one sentence. Closing the policy (with does it) gives its slot — or its place in line — to the next robot at once.

A slot follows the policy that holds it. While the policy is open, the SDK keeps the slot alive in the background, between acts too, so a robot that stops for a minute or two to reset its scene keeps it. It is given back the moment the policy closes (close(), or the end of a with), is garbage-collected, or its process exits or is stopped (SIGTERM, SIGHUP); a process that is killed or loses its network gives it back within 60 s. So is what ready() gives a robot before its first act — the slot it is told ready with, the worker a poll starts for it, its place in line — so a robot that gives up waiting frees them for the robots behind it at once, or within 60 s when it is killed. A robot cut off from the platform for up to about 30 s keeps its slot (up to 60 s when the cut comes just after a keepalive); one cut off for longer is told on stderr that its slot went, and its next act takes one again or waits in line. run() ending, or a Player's episode, does not give it back — the same policy can run again — so close the policy when the robot is done. Keeping a slot is not keeping a worker: a worker still stops idle_timeout after its last act (600 s by default), and its slots go with it, so a policy left open and unused costs at most that long. A robot whose slot went takes one again at its next act, or waits in line. When a worker has to start for it, run() and a Player wait for it before their first action and act() waits for it on every call, within startup_timeout_s; a Player's refill after its first row raises Unavailable ("the worker is still starting") rather than pause your loop mid-motion: call wait_until_ready(), then go on.

How big your frames are is how fast you go

Latency scales with the bytes you send, at roughly 16 ms per kB. This is the single largest thing under your control, and it is larger than the model:

observation    p50      mean     sustainable arms
14.6 kB      586 ms    623 ms         0.86        
 3.4 kB      384 ms    433 ms         1.23        

Measured alternating A/B against one warm worker over one connection, so that drift in the service cancels. inference_ms did not move between the two (119 ms against 111 ms) — every millisecond of the difference was transport. A direct measurement against the worker agreed: 181 ms for the same 11.2 kB.

Send JPEG, not arrays. A frame as a list of integers is roughly 16x the bytes of the same frame as base64 JPEG, and it is the most common way to land on the slow side of that table.

Use quality 95. Do not go below 90. With the sampling noise pinned, on a real DROID frame:

Quality 85 and below is where the error jumps by an order of magnitude. This section previously read "quality 85 is safe" and quoted the 0.51% figure next to it — but 0.51% is q95's number, and q85 is nearly four times worse. sequence_ai.encode() defaults to 95 so the decision is not yours to get wrong.

Binary frames: the frames' own bytes, not base64

From sequence-ai 0.36.1 a robot's act goes up as binary frames when the worker it is connected to reads them: each camera frame's JPEG / PNG bytes after the act's JSON, instead of base64 inside it — a quarter less to upload (pi05-droid's two 224x224 frames on dev: 33,911 bytes an act against 45,076 as JSON). The worker turns the bytes back into the very base64 strings you sent, so your policy reads exactly what it read before; there is nothing to change in your code.

• Only where the worker reads them. Each connection asks its own worker before its first act, and sends frames only while every answer says the worker reads them; otherwise its acts go as JSON, as before.

• A policy deployed before 2026-10-16 reads JSON only. A worker runs the harness its version was built with, so a version built before binary frames takes JSON until you seq deploy it again. seq policy status <name> says which: "its workers take a robot's frames as binary frames", or that a redeploy brings them.

• What a connection sends: policy.act_format is "frames", "json", or None until the worker has answered (wait_until_ready() waits for that). Why it is what it is goes to the debug log, once each:

  import logging
  logging.basicConfig(level=logging.DEBUG)   # or logging.getLogger("sequence_ai").setLevel(logging.DEBUG)
  

  the connection to https://…: a robot's acts go as JSON: its worker's answer to GET /ping (200) named no binary
  frames (X-Seq-Act-Formats: x-seq-frames/1) - a policy deployed before 2026-10-16 reads JSON only, until
  `seq deploy` builds it again
  

• Always JSON: a policy's own inputs (act({...})), an act with no frame binary frames can carry (none whose data is canonical base64), chat, an eval's acts, and every act with SEQ_ACT_FRAMES=0 — the switch for anything between your robot and its worker that mishandles a body that is not JSON.

Fit each frame on the robot, the way the policy was trained

A deployment declares the size each camera's frames arrive at (seq.Camera(width=224, height=224)), and a worker built on contract 0.5.2 checks every frame against it — it never resizes one. Fit the frame on the robot, before sending, with the tools the SDK gives you; nothing in the SDK resizes a frame behind your back (encode() sends a frame at its own size).

# the deployment says how (Camera(..., resize="pad", filter="bilinear")): read it from the contract
data = policy.prepare_image("exterior_1", frame)

# or name the size, the mode and the filter yourself
img = sequence_ai.resize_image(frame, 224, 224, mode="pad", filter="bilinear")
data = sequence_ai.encode(img)

mode="pad" keeps the aspect ratio and pads the rest black (openpi's resize_with_pad); "stretch" fills the box. The filter is one of nearest, box, bilinear, hamming, bicubic, lanczos. Neither has a default: they are how the policy was trained, and a frame fitted another way differs pixel by pixel. A deployment version built since the platform checks frame sizes refuses a frame that is not its camera's size: before it is sent, and on the worker if it is sent anyway — 400 invalid_request, details.sizes with each frame's size and the size its camera reads, and a hint with the call that fits it:

camera 'exterior_1' frame 0 is 640x360; the policy reads 224x224
fit each frame on the robot before sending: policy.prepare_image("exterior_1", frame), or sequence_ai.resize_image(frame, 224, 224, mode="pad", filter="bilinear")

A version built before passes such a frame to its policy as it is, so the client only warns, and says that deploying it again makes it refuse them. policy.contract["checks"] holds "frame_sizes" for a version that refuses them.

Fitted on the robot, a pi0.5 camera's 224x224 frame was 11.6 kB on the wire; the same frame sent at 640 px and resized on the worker, as encode() did until 0.34.12, was 44.9 kB.

Always JPEG. Measured in action space against the model's own call-to-call spread, quality 95 is indistinguishable from lossless; the cliff is far below, so there is no reason to spend 3-4x the payload on PNG — a native 1280x720 frame uploaded whole as PNG is 3.6 MB per view.

Fidelity below that is not a gentle slope. Degrading the frame further does not degrade the policy proportionally; it stops working. Quantising each channel to four levels — layout, edges, shadows and hues all preserved, a milder change than any sane JPEG setting — took pi0.5-DROID from placing the cube in the bowl in 5.3 s to never lifting it at all, with its closest approach 20x worse than the successful run's final distance. Measured in simulation, 2026-09-11.

What your deployment reads

Which cameras, how many frames of each, how long each proprioception and state vector is, whether it needs an instruction: that is what your policy declared (observation= on @seq.policy), and the worker refuses an act that does not match, saying what is missing. So the client checks first. The first request connects, which hands back the declaration, and every observation is checked against it before it is sent; frames beyond the declared history are dropped before upload.

obs = sequence_ai.observation(
    images={"exterior_1": cam.read(), "wrist_left": wrist.read()},   # ndarray or PIL
    joints=arm.joint_positions(), gripper=[arm.gripper_position()],
    instruction="pick up the cup",
)
answer = policy.act(obs)            # connects, then checks obs against what my-pi05 declares

A mismatch warns once and names it (missing view(s) ['wrist_left'] … refused); connect(validate="strict") raises InvalidRequest instead, "off" skips the check. From then on policy.contract holds the declaration, and observation(..., contract=policy.contract) checks while it builds: a camera the policy does not read is dropped with a warning that names it, and what the worker would refuse raises before anything is sent.

Each vector is checked against its own length. Eight joints and no gripper sums to the same eight numbers and is still wrong — the policy reads the gripper from its own field — so it is refused, not reshuffled.

What to do with the rows that come back

They are what your policy's infer() returned. What they mean is the policy's own declaration — action=seq.ActionContract(...): the robot, the action space, the dimensions, the rate — which the worker checks on every chunk, and which answer.action_chunk carries (action_space, robot, control_frequency_hz). Turning a row into a command for your controller is the robot's half: policy.to_command(step) clips each value into the deployment's bounds; policy.to_commands(answer) does the whole chunk. act() returns the model's own numbers, and run() (to step) and a Player (from next()) hand you a step already clipped into the bounds (and max_delta). A gripper the policy answers as a fraction is the template's own client.py to turn into open or closed. Execute open_loop_horizon rows (replan after N) before observing again: answer.rows stops there, and run() and a Player go past it only while the next chunk is late.

Three layers: run, Player, act

The same request underneath; the difference is who owns the loop. Start at the top, and take a layer apart only when you need what it holds.

policy.run(observe, step, max_seconds=30)   # the loop, ready-made
sequence_ai.Player(policy)                   # the loop taken apart, for a loop of your own
policy.act(obs)                              # one request: an Answer

run() observes, asks, plays each row at the policy's rate, fetches the next chunk while one plays, follows the platform's word on when to ask, counts the stalls, and opens and ends the episode. It needs a bound (max_seconds=, max_actions= or both); instruction= makes it the VLM's tool (above).

A Player is that loop with your own code in it. One with block is one episode, ended and reported when the block ends:

with sequence_ai.Player(policy) as player:
    while not done():
        if player.due():                 # time to ask for the next chunk
            player.ask(observe())        # on its way; returns at once
        robot.step(player.next())        # this tick's row

player.next(observe) asks by itself, at the moment run() would. next() waits for its tick and, with nothing in hand, for the next chunk; next(wait=False) never waits and returns None when it has no row, for a loop that keeps its own clock — call it at the rate the rows declare. Player(policy, max_age_s=2) plays no plan computed from an observation older than that: the rest of that answer is dropped (player.stats.stale_dropped) and the next one asked for. A policy whose infer takes feedback is sent, with each act, what the Player is playing as it goes; one whose actions declare aligned=True has each answer played from the row that is now, the rows that went by while it was inferred skipped (Player(policy, aligned=False) appends them instead).

act() is one request, and returns an Answer: answer.rows to execute and answer.spare past them, answer.outputs — what infer() returned, as it came (a policy's own inputs), answer.next_at — when the platform suggests sending the next act, in time.monotonic() seconds — and answer.sent_at, when this one went. policy.end_episode() ends the episode a loop of act(obs, episode_id=...) calls has open.

A worker seats the robots that say when they run dry and send when it tells them, staggering or grouping them on its copies; run() and a Player do both. A bare act() says neither and is served as it comes: give it steps_left= — the steps your robot still has to play — and send the next one at its answer.next_at to be seated.

A policy of its own inputs

A policy that declares no robot-arm action takes whatever observe() returns — anything but a sequence_ai.Observation — sent as it is, and answer.outputs is what its infer() returned, untouched. The steps of each answer are, in this order:

• what actions_of= returns, when you give one — it is handed what infer() returned;
• else the rows its actions=seq.Actions(key=...) names, one a step: the first open_loop_horizon of them before the next act, the rest played only while the next answer is late;
• else the default an eval reads them by too: a dict, or a list of numbers, is one action; a list of lists of one length, or a 2-D array, plays row by row; any other answer is refused, saying to declare actions= or give actions_of=.

They play at control_hz= (a number, or a function reading it from each answer), else at connect(control_hz=), else at the hz its seq.Actions declares. run(), a Player and answer.rows read them alike — act() only once you ask for answer.rows — and smooth_seam blends them only with absolute=True.

# the policy declares actions=seq.Actions(key="actions", hz=10, open_loop_horizon=8)
policy.run(lambda: {"depth": camera.read()}, arm.move, max_actions=500)
# or read the steps yourself, at your rate
policy.run(lambda: {"depth": camera.read()}, arm.move, max_actions=500,
           actions_of=lambda answer: answer["actions"], control_hz=10)

The other endpoints: chat and decide

Act is a loop; these are one-shot calls to models the platform serves itself (sequence_ai.models(kind="model") lists them; models() with no arguments — and seq models — also lists every template seq init writes, each marked kind="template" with its template_kind). Same key, same typed errors.

r = sequence_ai.chat("claude-opus-5-5", [{"role": "user", "content": "what should the arm do?"}])
r.content            # the reply — no choices[0] wrapper
r.tool_calls         # set instead when finish_reason == "tool_calls"

chat() is the slow brain: a sentence in, a sentence (or a tool call) out. It is answered by an external provider with no cold start. reasoning={"effort": "minimal"} and the other model parameters (seed, stop, top_p, …) pass through under their usual names (a model's supported_parameters in models() says which it takes); reasoning tokens are billed output and count against max_tokens. decide(model, state, questions) sends a state with typed questions and returns calibrated probabilities; help(sequence_ai.decide) shows the criteria each question type needs. Both results carry cost_usd. A request the provider rejects raises InvalidRequest in its own words; a provider error (502) or a rate limit / no answer (503) raises Unavailable; none is billed.

Benchmarks & eval

The other half of the SDK: instead of driving a robot, score a deployed policy on a set of simulation tasks. You write a benchmark the way you write a policy — a decorated class — register it with the same seq deploy, bridge its observations and actions to the policy with an adapter, and run seq eval run. Before spending GPUs, --dry-run shows the estimated wall-clock, the dollar cost, and how many GPUs you will actually be granted. The sims and the policy both run on the platform.

export SEQUENCES_API_KEY=seq_live_...          # identifies your account (owns deployments, billed for evals)
seq init toy-policy                            # toy-policy/     — a @seq.policy (policy.py, and client.py for a caller)
seq init toy-benchmark                         # toy-benchmark/  — a @seq.benchmark (benchmark.py)
seq init toy-adapter                           # toy-adapter/    — the @seq.adapter between them (adapter.py)

An adapter is the third kind you register, and the only one without a machine: it belongs to neither the policy nor the benchmark but to the pairing an eval runs. seq deploy registers it as a version — the same file again is the version it already is — and the eval names it (--adapter toy-adapter@v1, or toy-adapter for its latest); the platform runs it inside the benchmark's container, so changing it changes neither side's version.

An adapter says in its code which benchmark → policy pairs it converts between, and an eval runs it on those alone:

@seq.adapter(name="toy-adapter", pairs=[("template:toy-benchmark", "template:toy-policy")])
class ToyAdapter(seq.Adapter): ...

Each side is a name in your account (my-libero), template:<name> for anything made from that template, or * for any one; pairs="any" opens it to every pair. seq deploy refuses an adapter that declares none, and an eval whose benchmark and policy it does not declare is refused before anything runs, the error listing the pairs it does declare. Changing its pairs is a new version, like any change to its file.

seq deploy toy-policy                                     # a @seq.policy deploys, as toy-policy
seq deploy toy-benchmark --name toy-benchmark             # a @seq.benchmark registers (its folder holds a CPU one too)
seq deploy toy-adapter                                    # a @seq.adapter registers, as toy-adapter@v1 (nothing builds)
seq eval run toy-policy --benchmark toy-benchmark --adapter toy-adapter --envs 16 --dry-run
seq eval run toy-policy --benchmark toy-benchmark --adapter toy-adapter --envs 16

The same from Python — seq.deploy(Cls) does what seq deploy <file> does:

import seq

est = seq.eval("toy-policy", "toy-benchmark", adapter="toy-adapter", envs=16, dry_run=True)
est.wall_clock_s, est.cost_usd     # an estimate — it starts nothing and reserves nothing

report = seq.eval("toy-policy", "toy-benchmark", adapter="toy-adapter", envs=16)
report.metrics["success"].overall   # "success" is YOUR metric key from @seq.check — no platform success_rate
report.metrics["success"].per_task  # each task's value with its episode count
report.cost_usd         # actual dollars, reconciled from real GPU-seconds
report.dashboard_url    # the eval in the console
report.videos           # the recordings the platform made (if render= was declared)

--adapter ./toy-adapter/adapter.py:ToyAdapter registers a local file first. seq adapter ls lists your adapters, each with its latest version, its pairs and the eval that last ran it, seq adapter source toy-adapter@v1 writes its file back exactly as it was registered (seq.adapter_source("toy-adapter", 1) returns it) — edit it and seq deploy it again for the next version — and seq adapter check adapter.py lists every step as the platform will run it, without sending anything; seq adapter check adapter.py --benchmark toy-benchmark --policy toy-policy also runs a sample of each side through it and says where they do not fit, with no eval and no GPU.

seq eval status <id> says an eval's status (queued for GPUs, running, completed, failed or canceled) and seq eval cancel <id> stops it; seq eval report <id> [--wait] [--videos DIR] [--csv [-o PATH]] reads it later - with, under the policy's load, what the eval's own measurements say is worth declaring (a hint: line), as the console says it in its Load's tip; seq eval ls <policy> lists a deployment's evals over time, 50 a page (--limit N, and --cursor C for the next page, as the page's last line says — every ls pages this way, the same pages as the console).

You lay the eval out; the platform never picks the numbers. The policy runs on its own machines (--policy-machines, on the deployment's cards unless --policy-gpu names one) and the benchmark on its own GPU machines (--benchmark-machines, each on an L4 or an RTX-PRO-6000 — or a CPU machine for a gpu="cpu" benchmark — running --envs environments at once); unset, each is one machine. --spend-cap USD is the eval's spend cap, and --max-steps N its step limit. Every card counts against your account's limit: past it the eval is refused, and while your cards are busy the whole eval waits for them (up to 10 minutes, then it is canceled; nothing is held or billed while it waits, and seq eval status <id> shows the wait) — never a silently smaller layout. --dry-run shows the layout, the cards against your account (what serving and evals hold, what is free, how many short and why) and the estimate. --sweep adds arms to the same eval, run side by side on the same episodes. If the policy and benchmark disagree on obs/action shape and you pass no adapter, the eval still starts, with a warning naming the adapter to run it with; if they really do not fit, it stops at its first act (stop reason mismatch, billed only for what ran) rather than handing you a silently-wrong score.

The full reference — the benchmark lifecycle (@seq.load / @seq.setup / @seq.reset / @seq.check / @seq.score, the platform-owned reset→step→check loop), assets= + render=, adapter loss labelling, billing, composite (VLM) evals, and every seq.eval parameter — is in eval.md.

Safety

An action returned by any model is model output, not a safe robot command.

This library does not check joint limits, reachability, collisions, or whether a step is safe at the robot's current velocity. Bounds checking, a watchdog and an e-stop belong between this library and your motors.

policy.run(
    observe=robot.read_observation,
    step=robot.apply_action,
    validate=robot.is_safe,   # return False to stop the loop
    hold=robot.hold,          # called if it stops early, or if step() raises
    max_actions=250,
)

A bound is required — max_actions=, max_seconds= or both, keyword-only. There is no run_forever() — an unbounded loop that moves a robot should not be startable by accident.

validate=None is allowed for bench and simulation work, and warns once so it cannot happen silently on real hardware.

Declared limits are enforced twice. An action contract can carry bounds (each dimension's [low, high]) and max_delta (the largest change between consecutive steps): seq.ActionContract(..., bounds=[(-0.05, 0.05)] * 6 + [(-1, 1)], max_delta=0.02). The worker clips every chunk into them before it leaves — the response's warnings name each clipped step and dimension. The robot client clips again before every action it hands out, measuring max_delta from the previous action even across chunks (a new run() starts afresh). They narrow what reaches the motors; they are not a substitute for the checks above.

Errors are typed

Every error the platform sends has one shape — detail, code, retryable, retry_after_s, request_id, hint, details — and becomes one exception class, the same from chat(), connect() / act(), seq.deploy(...) and every seq command. Switch on .code, show .hint, quote .request_id when you report a problem.

A failed seq command prints error[<code>]: <detail>, then hint: … when there is one (and request_id: … for a 500), and exits with the number in the last column — 0 is success, and seq eval run --gate exits 8 when a metric misses its bar — so a script can tell "retry" from "give up" without reading the text.

Debugging a deployment

When a deployment misbehaves — some control requests slower than expected, or one that just failed — two read-only commands localize the fault and pull the record for a single request by its id:

seq policy doctor <deployment>              # probe the direct-connect chain segment by segment
seq policy logs   <deployment>              # structured logs, connect + worker sinks merged, time-ordered
seq policy logs   <deployment> --rid r-…    # one request: worker phase joined to client timing, attributed

seq policy doctor <name> runs on the platform, segment by segment: the deployment's state and serving version, a running worker's /ping, and one /act on a synthetic observation built from the contract you declared (black frames, zero vectors) — with the round trip split into network vs. worker-handler, so you can tell a slow network from slow inference. It never starts a worker: a deployment scaled to zero says so.

seq policy logs <name> shows your workers' own lines — the model load, every act with its request id and timings, every error with its reason — beside one line per connect and the timings your robots' clients reported. seq policy logs <name> --rid r-… joins one request across all three. Workers built after 2026-10-02 send their lines; an older deployment shows them once deployed again. Lines are kept seven days. Times print on this machine's clock with the zone's name (14:05:09 EDT, or a ── Mon 2026-10-12 · EDT ── line above a day's lines); usage and bills - seq usage, a policy's monthly budget, your plan's credit - go by the billing period, the UTC month, and say UTC. --json gives the platform's own timestamps. Authorization is enforced server-side against your SEQUENCES_API_KEY; you only see your own deployments. For the account side, seq account (sequence_ai.account()) shows your balance, usage, limits and GPUs.

Full walkthrough, the segment catalog and the redaction guarantee are in observability.md.

Authoring & deploying your own policy

Everything above drives a deployment. To make one — declare the environment your policy needs, point it at your weights, and have the platform validate that environment — use seq: the seq import and the seq command. They come with the same pip install sequence-ai on Python 3.10+; on a 3.9 robot that install is httpx and Pillow only, without seq.

A policy is a class decorated with @seq.policy. You declare what it needs — the GPU, the image, where the weights live — and mark its lifecycle methods; nothing is built or downloaded on your machine, the declaration is pure data the gateway builds from.

import seq

@seq.policy(
    gpu="L4",                                  # the one card `seq gpus` lists ("H100:2" is two of them), or "cpu";
                                               # a start waits for it and never switches to another
    max_gpus=1,                                # the most cards it holds at once: required on a GPU
    cpu=4, memory=16,                          # cores and GiB requested, or (request, limit); undeclared: the
                                               # minimum, 0.125 / 0.125, and the worker may use more
    image=(
        seq.Image.debian_slim()                # a concrete Debian base, pinned
        .apt_install("git", "libgl1", "libglib2.0-0")   # system libraries
        .run_commands(                         # openpi installs from source: its PyPI name is a placeholder
            "git clone https://github.com/Physical-Intelligence/openpi /opt/openpi",
            "cd /opt/openpi && uv pip install --system -e .",
        )
    ),
    weights=seq.Volume.from_name("pi05-ft"),   # your checkpoint: an uploaded volume, hf://, or a template's (below)
)
class Pi05(seq.Policy):
    @seq.load
    def load(self, weights_dir):               # cold start: pull the checkpoint into VRAM
        self.model = ...

    @seq.infer
    def infer(self, obs, *, seed=None):        # per step: what the robot sends -> what it gets back
        return ...

seq deploy [folder] (this one, ., by default) uploads the folder, whole — its helpers, configs and third_party/ with it — and that snapshot is the version's code: nothing of an earlier version is kept. The class it deploys is found by reading the folder's files, never by running them: its one @seq.policy, @seq.benchmark or @seq.adapter class (an adapter beside a policy or a benchmark is not the one); of several, --name picks one by its name. The class is imported by its path in the folder (policy/OpenWAM/model.py → policy.OpenWAM.model), the folder first on sys.path and the class's own folder second, as running its file would — on the platform too — and a module it imports from outside the folder is refused, with the ways to bring it in. Left out, and named in the output: version control and caches anywhere; a virtualenv or a build's output at the folder's root (.venv/, venv/, env/, build/, dist/, node_modules/, *.egg-info); a .env file — secrets go through seq secret, and a !.env line uploads it; and what a .seqignore lists — one in any folder, for that folder down, in gitignore's form: a path from its folder (third_party/openpi/docs), a name at any depth (data), * and **, a trailing / for directories only, and ! to take back what a line before it left out. The deploy prints how many files, how large, and their tree sha256 — the hash the platform keeps beside the version (seq deploy checks the two agree). A path the image names with add_local_file / add_local_dir inside the folder is the folder's copy, uploaded once. Weights never travel with the code, only through weights=.

A template's third_party/ is the source its image installs: change it and deploy again, and your deployment runs your version. An upstream too large to upload with a deploy (LIBERO, robosuite) is listed in third_party/SOURCES.json: seq init downloads its official package there and checks it (seq init --no-source leaves it out), the image installs that same package by its sha256, and seq deploy refuses a copy you changed — a large upstream runs as published for now.

The image is declarative

Every seq.Image method returns a new image and builds nothing — the chain is a spec, and the platform builds it into a layer. Start from one base, then extend it; the step order you write is the build order, one for one.

python= picks the worker's Python: "3.11" (the default), "3.12" or "3.13", or "image" for the one the base already has (it must be 3.11 or later). An official python:* base brings its own; a Dockerfile brings its own unless you pass python=.

A build step that needs a credential — a private git repo, a private package index — declares it on that step: .run_commands("pip install git+https://$GH_TOKEN@github.com/acme/private", secrets=[seq.Secret.from_name("gh")]). Only that step sees GH_TOKEN, only while it runs; no image layer and no worker holds it.

The platform's own runtime packages (an HTTP server, pydantic, numpy, pillow) go in right after the base, before your steps — so a version you pin (.uv_pip_install("numpy==1.26.4")) is the one your policy runs with.

@seq.policy(deps=[...]) is shorthand for a trailing .uv_pip_install(...), so a quick policy needs no explicit image= at all. Build-time env carries ordinary variables only — a credential put there is rejected, because the image is not a place for secrets (see below).

Where it runs, how long it may take, and what the build is doing

• gpu="L4" is the one card the deployment runs on. A start waits for it to come free - up to start_wait_s, 10 minutes unless seq policy scale <name> --start-wait SECONDS sets another (60-3600) - and never switches to another card; past it the robot is told which card and how long it waited. A list runs on its first card.
• Workers run wherever a card is free; a policy does not choose where.
• timeouts=seq.Timeouts(startup=900, request=5): a worker that has not loaded within startup seconds fails (the build check is held to the same), and one inference longer than request seconds is answered 504 while the worker finishes it before serving again. Defaults: 1800 and 300.
• seq deploy shows each phase of the build as it begins — the image, the weights, the worker loading, its snapshot, the /act check — with the newest line of the build's log; seq policy logs <name> --build prints the newest version's phases and its whole log, then or later (seq policy logs <name>@v3 --build: version 3's).
• seq secret set <name> reaches deployments that declare the secret without a re-deploy: their next workers start with the new value (running ones keep the old one until they are replaced).
• models=["qwen3-8-27b"] names the platform models the policy calls while it runs (a planner's VLM, a decide model). The robot key that drives the deployment may call exactly these on /v1/chat and /v1/decide — any other model is refused 403, and a full key is never needed on the robot or the worker. seq deploy refuses a name the catalogue does not have (seq models lists them). These calls, and the catalogue they look a name up in, are the network a deployed policy can count on: any other address — a model service or planner of your own, a download at run time — may not be reachable from its worker, so put what the policy needs into its image or weights.

Before it serves: one real /act

A policy that declares observation= is proven before its first robot: the build ends by calling its /act once on a synthetic observation — black frames at each declared camera size, zero vectors of the declared lengths, an instruction. A policy that cannot answer it fails the build, and seq deploy shows the policy's own error instead of a robot finding it later. Without an observation contract there is nothing valid to send, so only the load is checked.

Knobs a request may turn: inference parameters

A keyword of @seq.infer with a default is an inference parameter: def infer(self, obs, *, seed=None, replan_steps: int = 5) declares replan_steps. Its type is the annotation (or the default's): int, float, bool or str. seq deploy --dry-run lists them; a robot sets one per call with policy.act(obs, params={"replan_steps": 3}) (run(..., params=...) for every act of a run), and an eval with seq eval run … --param replan_steps=3 (or --sweep replan_steps=2,5 to compare values). The worker casts each to its declared type and refuses, before any inference, a name the policy does not declare (400 params). Unset, the default applies. An eval sets an adapter's __init__ keywords and a benchmark's @seq.setup keywords the same way, plus its own eval.steps_per_query — see Parameters in an eval.

State across steps, and more than one robot at a time

• What a policy keeps of an episode is read from how it is written. An @seq.infer that takes episode: seq.Episode is handed it: episode.state is a dict of its own, empty on the episode's first act and the same object on every act after, dropped by the platform at the episode's end — so one copy serves several robots in turn. A class that keeps a robot's state on itself — a KV cache, a frame window — frees it in an @seq.on_episode method (or names reset= / end=), and each episode then holds a copy of its own. policy.run() ends the episode when it returns, even on an exception, and a Player when its with block ends; a loop of policy.act(..., episode_id=...) calls ends its episode with policy.end_episode(), and close() ends one left open; an episode with no act for 5 minutes (a robot that crashed) is ended by the worker (seq.Timeouts(episode_idle=...) sets another time). A policy that reads every observation since its last act (observations=seq.Observations(every=1)) gets each one as the robot makes it: policy.observe(obs, episode_id=...) hands it over at once - posted to the worker on a connection of its own; a first call, the one that connects, waits out a worker that is starting as a first act() does, and one made while its worker restarts mid-episode is kept for the next act - and the episode's next act(newest, episode_id=...) carries only its own. A policy that declares a window of steps (observations=seq.Observations({"cam": seq.Stream(at=[-15, 0])})) has its worker keep the frames its acts read: hand it every step's observation (run() and a Player do), only the steps a window reads go up, and each act is handed them with its own. Whatever it declares, even each act's own step alone, infer is handed {"observations": [...], "steps": [...]} with only the keys it declares, on a robot as in an eval - each act's step a Player's rows played, else the next of its episode. When the copy holding an episode stops (out of memory, a crash), the robot goes on in a new episode with a warning - its steps counted from 0 again, none of the old one's observations handed to it - and run() and a Player count it in episode_resets; on a worker of one copy the act sent again finds it still starting - a loop of act() calls waits that out, while run() and a Player raise it, saying so, and run() called again goes on in the new episode. A policy that only needs the last N frames needs no state: declare observation=seq.ObservationContract(history=N) and the robot sends them.
• What the robot was playing. An @seq.infer that takes feedback: seq.Feedback is handed with each act what the robot was playing when it sent it: left, the rows it had not played yet; delay, the steps it expects to play before the answer is back (the longest of its last 10 round trips at its rate, rounded up; 0 in an eval); played, the rows of its chunk it had played; and last, the row it played last. A Player, run() and an eval send it, and act(obs, feedback={...}) sends your own; under @seq.concurrent it is a list, one for each observation. aligned=True on seq.ActionContract or seq.Actions says a chunk's first row is the moment of its observation: a Player takes each answer as it lands and plays it from the row that is now, at lookahead=1.
• A worker has seats; a robot holds one for its whole control loop. A worker runs one copy of the model on each card group, each copy its own process. @seq.policy(gpus_per_replica=N) gives one copy N cards (it must divide gpu="TYPE:N"'s N); by default one copy sees every card of its worker and splits them itself. Without more, a copy serves one robot at a time. @seq.concurrent(max_concurrent=N) on the @seq.infer method lets a copy run up to N acts in one forward pass — infer then takes a list and returns a list of chunks in the same order. seq.collate(obs, buckets=[1, 2, 4, 8]) stacks that list into one batch — each array along a new first axis, a first axis that differs padded with <key>_mask beside it, the batch padded to a size the model was compiled for — seq.uncollate(out, n) splits the answer back, and group_by= keeps requests whose parameters differ apart. A call that raises is run again act by act, so an act gets an error only when it fails on its own. An infer that takes episode beside it is handed episode: list[seq.Episode], one for each observation in order, and a batch holds at most one act of an episode; when such a call raises, only the acts of episodes that had none before on that copy are run again alone, on an empty state, and the other episodes end - their state may be half-written - so their robots go on in new ones. With dynamic=True (the default) each worker measures its forwards and seats as many robots as it can serve on time, telling each when to send: apart while each can be served alone, together only when they cannot. It plans the robots that say when they run dry and send when told — run() and a Player do both; a loop of act() calls gives steps_left= and sends at answer.next_at (Three layers). With dynamic=False each worker seats N robots and runs their acts together. The older @seq.batched, max_batch_size, max_inputs, target_inputs and the batch waits are refused, each with what to write instead. Several copies on one card (replicas_per_gpu=) are gone since 15 October 2026: two pi05 copies on an L4 made each act nearly twice as long when both acted, and a 15 Hz robot executing 8 steps a chunk ran dry.
• More workers, warm workers — limits count GPUs, never workers. A deployment starts another worker whenever every slot is taken, until it reaches max_gpus=N (the most cards it may hold at once; a GPU deployment must declare it, or deploy with --max-gpus N) or your account's GPU limit. A robot that arrives when every slot is taken and no GPU is left gets on_full: "queue" (the default) puts it in line before its control loop starts — connect() waits it out and prints queued for a GPU slot: position 2, … — for at most queue_timeout_s (default 120); "reject" refuses it at once. A loop that has started is never interrupted by a robot arriving, and closing the policy gives its slot to the next in line at once. min_workers=1 keeps that many workers running with no robot connected (no cold start, billed while they run); buffer_workers=1 keeps one spare while robots are connected; idle_timeout=300 is how long an idle worker stays (default 600 s). seq policy scale <name> --max-gpus 4 --on-full reject --min 1 … changes any of these at once, without a redeploy, and seq policy status <name> shows each worker's copies of the model, the robots it seats and those on it, the robots in line, and why a worker died - and, under the figure it comes from, what the deployment's own measurements say is worth adding next (a hint: line), as the console says it in that figure's tip.

GPUs, limits and what they cost

seq gpus lists the cards gpu= can name — memory, price per hour and per second, whether a policy and a benchmark may use it — and what a CPU core and a GiB of memory cost (sequence_ai.gpus() from Python). Your account may hold 8 GPUs at once, serving and evals together; seq account shows how many each holds now, how many are left and which evals are waiting for GPUs (a count the platform could not read says unknown, never full).

GPU, CPU and memory are billed apart, per second. A card is billed per card, for at least 60 s each time a machine starts. CPU and memory are billed on the greater of what the worker requests and what it actually uses: cpu= / memory= is the request, and without them a worker requests only the minimum (0.125 cores, 0.125 GiB) and may use more — you then pay for what it uses. Write cpu=(2, 4) to cap it; declare memory= for a policy that runs a copy on each of several cards or holds large buffers, so the worker is not squeezed to the minimum. seq deploy prints one worker's price per hour in three parts once it is ready. seq usage shows what your machines used and cost over this billing period (the UTC month so far), per hour or day in UTC: GPU card-hours, and dollars for the GPU, CPU and memory together. --policy <name> keeps one deployment's rows and adds what its robots did, --since / --until name another window, and --csv prints the rows alone.

When there is no GPU for a request, the error says which of five things happened, in one sentence: your account's GPUs are in use (with what serving and evals hold), every GPU on the platform is busy, no card of the type you declared came free within the deployment's start wait (seq policy scale <name> --start-wait SECONDS lets it wait longer), the deployment holds its max_gpus, or a queued robot waited past queue_timeout_s. details["reason"] carries the same as a code (gpu_limit_reached, gpu_busy, gpu_low_stock, deployment_full, queue_timeout).

A model on more than one machine

Write the total: @seq.policy(gpu="H100:16"). Past 8 cards the platform runs the model on whole machines of 8 cards — 16 is two machines, at most 256 cards — started together, stopped together and billed together (every machine's cards, CPU and memory). The account needs a GPU limit that holds the whole group at once; past the 8 an account has by default, ask us.

• Your @seq.load and @seq.infer run on every machine, one process each, seeing that machine's 8 cards. Before your code is imported the platform sets RANK, WORLD_SIZE, MASTER_ADDR, MASTER_PORT and LOCAL_RANK=0 and, when your image has torch, initializes torch.distributed (NCCL) across the machines — check torch.distributed.is_initialized() rather than initializing it again. Code on another framework reads the same variables to join its own, on a port other than MASTER_PORT.
• A robot reaches machine 0. Every request — and every episode end — runs on every machine at once, so the collectives in your code meet; machine 0's answer goes back to the robot.
• The group is one copy of the model and holds one robot's control loop: leave gpus_per_replica unset, list one card type, and keep a policy with @seq.plan on one machine.
• When one machine stops, the whole group stops (group_member_lost) and the next connect starts a new one. seq policy status <name> shows each machine's rank, its state and when it last answered.

Weights live on a volume, never in the image

weights= is copied once, at deploy, into a volume the platform mounts at /data — never baked into an image layer (so a multi-GB checkpoint never bloats the build), and every cold start reads that copy. It names one of three things:

• An official template's weights — seq.Weights.template("pi05-droid", version=1) (the string form is "template://pi05-droid@v1"): the unmodified checkpoint a template ships with, plus what it loads at runtime (pi0.5's tokenizer). A published version never changes; leave version= out to take the newest one at each deploy. No key is needed. This is what seq init pi05-droid writes.

• Hugging Face — hf://org/model or hf://org/model@<revision>. At deploy the revision is resolved to a commit and exactly that commit is copied, so a re-deploy is deterministic. A private or gated repo needs a token: store it once (seq secret set hf --env HF_TOKEN) and declare it, secrets=[seq.Secret.from_name("hf")]. The copy runs with only that token; declaring it also hands HF_TOKEN to your policy at runtime. A repo the platform cannot read — private without a token, gated, a wrong revision — is refused when you deploy, in seconds, with what to fix.

• Your own files — upload a local checkpoint directory once, then deploy it by name:

  seq volume put pi05-ft ./checkpoints/pi05-ft     # resumable: run it again after an interruption
  seq volume ls                                    # name, state, version, size (50 a page: --limit, --cursor)
  

  @seq.policy(weights=seq.Volume.from_name("pi05-ft"), ...)    # or weights="vol://pi05-ft@v2" to pin
  

  Every file's md5 is checked when the upload completes; only then does the version become active (and billed as storage). A new put of the same name makes the next version. seq volume rm is refused while a deployment's weights, or a registered benchmark's assets, use the volume. Deleted a volume, or replaced a version with a new put, by mistake? Write to founders@generalsequences.com within 7 days and we can bring it back.

Weights from your own storage bucket or an https URL are not supported. Move the files into a volume once — download them, then seq volume put <name> <dir> — and deploy weights=seq.Volume.from_name("<name>"), or point at the Hugging Face repo they came from.

Weights are validated as you write them: an unknown scheme, a .. traversal, an hf:// without a repo, or a template name that is not one is rejected with a message that names the problem.

seq.Secret.from_name(name) references a credential by name; the plaintext never enters the SDK or the deploy payload.

Versions, rollback, stop

Deploying a name that already serves never takes it down: the new version builds beside the serving one, robots keep acting on the old version the whole time, and the new one takes over only once its build is ready. A version that fails to build changes nothing — the old one goes on serving, and seq deploy says so. New sessions go to the new version at once; a robot already acting moves to it at its next episode boundary (or its next lease renewal, between episodes), onto a worker started for it in advance.

seq policy ls                        # your deployments: the serving version, one building beside it, the card
                                     #   (50 a page; the last line names the --cursor of the next)
seq policy versions pi05-ft          # v4 building · v3 serving · v2, v1 kept — each with its build
seq policy rollback pi05-ft          # serve the previous version again — instantly, nothing is rebuilt
seq policy rollback pi05-ft@v1       # or a specific kept version: a version is always NAME@vN
seq policy stop pi05-ft              # emergency stop: every worker ends now, every connect is refused
seq policy start pi05-ft             # let robots connect again
seq policy rm pi05-ft                # delete it: stops its workers now, frees the name and the quota slot
                                     #   (refused while robots are connected: --force disconnects them);
                                     #   it lists first what goes with it, and --dry-run lists it alone

The serving version and four earlier ready ones are kept; a sixth pushes the oldest out.

A stop ends workers, never a build: a version still building finishes and is checked as usual, its check billed like any worker, and once it passes it is the version that serves when you seq policy start the deployment.

A canary. seq policy traffic <name>@v5=10 <name>@v4=90 sends one new session in ten to v5 and the rest to v4 (any versions seq policy versions lists as serving or kept); seq policy traffic <name> shows the split, --clear sends everything to the serving version again. A session keeps the version it started on for its lease renewals, so a robot never switches versions mid-run. To try a version on one robot first, pin that robot's key (a key bound to this deployment) to it: seq policy traffic <name>@v5 --pin arm-7; --unpin arm-7 lets it follow the split again. A pin reaches every gateway within five minutes.

Budget, schedule, alerts, and the price before you deploy

• seq deploy <folder> --dry-run [--hours-per-day 8] prices the policy's card from the platform's list once the platform has checked it: its warm floor around the clock, or the hours a scale-to-zero deployment is in use. Nothing is built.
• seq policy budget <name> --monthly 100: when the month's spend reaches it, the deployment stops — workers end, connects are refused and say why. To start it again, raise the budget (or --off), then seq policy start <name>; under a budget already reached, seq policy start is refused.
• seq policy scale <name> --min 2 --during "Mon-Fri 08:00-18:00" --tz America/Los_Angeles keeps two workers warm in those hours (billed whether or not a robot is connected); --clear removes it.
• seq policy alerts <name> --webhook https://… [--error-rate 0.05] [--p95-ms 500] [--budget-pct 80] posts JSON to your https endpoint when a version fails to build, the hour's error rate or p95 crosses the threshold, or the budget nears or is reached. Each body is signed: X-Seq-Signature: sha256=HMAC(secret, body) with the secret printed once.

Field data: capture, label, export

seq policy capture <name> --rate 0.1 [--images] makes the deployment's workers keep a tenth of its episodes (each kept whole: the observation it sent, the chunk it got, the inference parameters), with the camera frames when you ask. No redeploy: running workers pick it up within a minute. A robot marks how an episode went with policy.label(success=True, notes="") after its run(). seq policy data export <name> --format lerobot -o DIR writes a LeRobot v2.1 dataset (one frame per /act, the chunk's first action, the episode's label as next.success; needs pip install 'sequence-ai[data]'), or --format jsonl the files as kept. Kept data is billed as the capture:<name> dataset volume, removed after --retention-days (default 30), and deleted with the deployment. seq policy capture <name> --off stops it.

Have the platform validate it

seq init pi05-droid my-pi05                # scaffold a working template to edit
seq deploy my-pi05 --dry-run              # a folder of several classes:  seq deploy my-folder --name pi05 --dry-run

seq deploy --dry-run reads the policy's declaration — it never runs load or infer — into the deployment request, validates it against this SDK's copy of the platform's contract, and echoes back exactly what will be installed, where the weights mount and which files are uploaded — with nothing built or run. It says, as seq deploy does, what the declarations make of the policy: what it reads of each observation and how its answers play (a policy of its own I/O), whose its episodes are, what its build check acts on, and how many robots a copy seats - and says once what to add when a policy that could seat several robots a copy (@seq.concurrent, or an infer handed its seq.Episode) seats one because nothing times it: a sample episode (sample=), or an observation contract. With SEQUENCES_API_KEY set, the platform checks it too, says whether it would be a new deployment or a new version of yours - and for a name deployed before, its last first start: how long its newest measured version took to ready, step by step, and about what this version's check costs on its card (a name with no version measured yet says there is no data) - and the dry run prices it; without a key it is the checks here alone, and nothing is sent:

policy      my-pi05
contract    valid for this SDK's deployment contract 0.5.8 (the platform checks it again when you deploy)
image       nvidia/cuda:12.2.2-cudnn8-runtime-ubuntu22.04@sha256:131e238d724ee145317f10d6c8eba0d301439c6c8764b02473510e7035756e81
  add               third_party/openpi/src → /opt/openpi/src
  add               third_party/openpi/packages/openpi-client → /opt/openpi/packages/openpi-client
  run               python -m pip install --no-cache-dir --no-index --find-links /seq/frozen/wheels --no-deps --require-hashes -r /seq/frozen/requirements-policy.lock
  run               python -m pip install --no-cache-dir --no-index --find-links /seq/frozen/wheels --no-deps -e /opt/openpi/packages/openpi-client -e /opt/openpi
  env               OPENPI_DATA_HOME=/data/.cache/openpi
weights     template://pi05-droid@v1
            → mounted at /data  (provisioned by the platform onto a volume, not into the image)
code        the folder ./my-pi05, whole: 85 file(s), 1.0 MB (0.31 MB compressed)
            the class it deploys is in policy.py
            .seqignore, CHANGELOG.md, LICENSE, NOTICE, README.md, client.py, frozen.toml, pins-policy.txt, policy.py, requirements-policy.lock, third_party/ (74), upstream.toml
            tree sha256 92c5890ac769050b5e0cd6569347fd095e8205800e132d5dc0baf0f9ffbbbc3f
            left out: third_party/openpi/.github/ (.seqignore: third_party/openpi/.github)
                      third_party/openpi/docs/ (.seqignore: third_party/openpi/docs)
action      droid_franka · joint_velocity · 8 dims · 15 Hz · bounds [-1, 1] ×7, [0, 1] (each step clipped into them)  (every chunk is checked against this)
observation exterior_1 224x224 (pad · bilinear), wrist_left 224x224 (pad · bilinear) · joint_positions[7] · gripper[1] · instruction  (every /act is checked against this)
episodes    none kept: any copy answers any act
check       acts on an observation built from its observation contract
seats       as many robots a copy as its forward times fit, up to 16 acts a forward - timed on an observation of its contract before it serves

name        'my-pi05' (not checked: the platform could not be asked)
platform    not checked: the platform could not be asked (no key, offline, or a platform from before the check); it checks this when you deploy

--dry-run: nothing was sent; without it, `seq deploy` submits this request.

(Shortened here: the template's image has a few more steps, and its .seqignore leaves out eleven more paths.)

A benchmark and an adapter are checked the same way: seq deploy my-reach --dry-run has the platform check the registration — a benchmark's asset volumes and secrets (each yours, uploaded), its card, cores and memory, and your benchmark limit; an adapter's name (one another class holds needs --replace) and its pairs — and says whether it would be a new one or the next version of yours, and for a benchmark its last first start, as for a policy. A refusal ends the dry run as it would end the registration, and nothing is registered or built; without a key it is the checks here alone.

The folder goes up whole, the robot's client.py and the README with it, less what its .seqignore files list.

Without --dry-run, seq deploy my-pi05 hands the build to the platform (it needs SEQUENCES_API_KEY). A malformed declaration fails here, at authoring time, with a readable error and a non-zero exit — not as an opaque build failure minutes later.

Configuration

export SEQUENCES_API_KEY=seq_live_...      # or pass api_key= to connect()
export SEQ_ACT_FRAMES=0                    # optional: every act as JSON, never binary frames (see "Binary frames")

Get a key at app.generalsequences.com. A robot connects with its policy's robot key instead - seq_robot_…, made on the policy's Robot keys tab - which reaches that one policy only; a robot key made before has seq_live_ and works until it expires.

Secrets — bring your own keys (BYOK)

Some policies need a credential you own: a Hugging Face token for private hf:// weights, or the key of a model service of your own that a policy calls itself. A composite policy's planner needs none: by default it calls the platform's own models through /v1/chat, with a short-lived credential the platform hands the policy, and each call is billed to your account like any chat call; point it at a service of your own instead and that service bills you. You store those keys once, by name, and refer to them from a policy by that name alone — the key value never goes into your code, your logs, or any output.

A secret is one name bound to a set of environment variables. The two canonical bundles are expressible by name alone:

seq secret set hf      --env HF_TOKEN            # private hf:// weights (and your policy's own use)
seq secret set planner --env PLANNER_API_KEY     # a composite policy's planner on a model service of your own

Values are read from stdin, never the command line

There is no --env KEY=VALUE and no --value flag — a value never lands in argv, your shell history, or the process listing. Pipe it in, or type it at the hidden prompt:

printf '%s' "$PLANNER_API_KEY" | seq secret set planner --env PLANNER_API_KEY
# or, interactively, you are prompted per key with the input hidden:
seq secret set planner --env PLANNER_API_KEY
#   Value for PLANNER_API_KEY (input hidden):

A bundle with several env vars reads one line per --env, in order:

printf '%s\n%s\n' "$WANDB_API_KEY" "$WANDB_ENTITY" | seq secret set wandb --env WANDB_API_KEY --env WANDB_ENTITY

List and remove — names only, never values

seq secret ls          # name + env-var NAMES + timestamps; the value is never shown, ever
seq secret rm planner

ls is metadata only. There is no command that reads a stored value back — from the SDK a secret is write-only. Re-running seq secret set <name> rotates the key (it replaces the whole bundle); no delete first. A rotated value goes live on the next worker start; seq policy stop <name> then seq policy start <name> replaces the running workers at once.

Use it from a policy — by name only

import seq

@seq.policy(gpu="L4", max_gpus=1, secrets=[seq.Secret.from_name("planner")])
class MyPolicy(seq.Policy):
    ...

Secret.from_name carries the name and nothing else — no value parameter exists, and it does no network I/O. seq deploy checks at deploy time that the name exists and is yours.

set/ls/rm each require your key (export SEQUENCES_API_KEY=seq_live_...) and are scoped to it: another key never sees your secrets. A missing or invalid key fails closed — nothing is stored, listed, or deleted.

One install, two halves

pip install sequence-ai on Python 3.10+ brings everything: the robot client (sequence_ai), the seq command and the decorators, with pydantic and the shared contract package sequence-base. On Python 3.9 — a robot's interpreter — the same install is the client alone, and seq says it needs 3.10+ instead of a stack trace.

Limits

What the platform fixes; everything else is your code's. Each value is what the platform runs today.

Install footprint

On a robot (Python 3.9), two dependencies: httpx, and Pillow, which fits and encodes each camera frame on the robot (resize_image, Policy.prepare_image, encode). On Python 3.10+, seq adds pydantic and sequence-base.

Extras add what some commands need: data (pyarrow, for the LeRobot export) and port, what moving a CPU benchmark to the GPU runs on (seq port export, check-model, scaffold, compare): MuJoCo 3.15.0, MuJoCo Warp 3.15.0 and SAPIEN 3.0.3, the versions the full test suite passes on. pip install "sequence-ai[port]" (Python 3.10+; on Linux aarch64, MuJoCo Warp needs glibc 2.34 or later and SAPIEN, which has no wheel there, is left out). A benchmark that runs on another MuJoCo version installs that version instead, as seq port export asks.

A robot controller is often on a Jetson with a pinned, fragile Python environment, and ROS 2 Humble ships Python 3.10. Every transitive dependency is another chance for the install to fail on the machine that actually matters. Pillow is the one the robot's own work needs: it has wheels for every platform a robot runs (Linux aarch64 included), and an environment pinned below Pillow 10 is upgraded by the install.

Running the tests

pip install -e ".[dev,port]"
pytest

Nothing is skipped. Without port, the three port test files are skipped whole when they are collected (32 tests), and a run that looks complete is not; on Linux aarch64, where SAPIEN does not install, its file is the one skipped.