positronic-platform-client

The wire contract for the Positronic evaluation platform, and a thin HTTP client over it: request and response models, id and enum types, the error envelope, and one PlatformClient method per API endpoint.

Alpha, under rapid development. Names, fields, endpoints and behaviour change without notice, and nothing here is covered by a backwards-compatibility guarantee. Pin the exact version you tested against, and expect to edit your code when you move off it.

The library depends on pydantic, httpx, typing-extensions and positronic-evals, which holds the eval names and definitions (evals/). So a service that only speaks to the platform installs it on its own, at the exact version it was written against:

uv add "positronic-platform-client==0.37.1"
uv add "positronic-platform-client @ git+https://github.com/Positronic-Robotics/positronic@<tag or commit>#subdirectory=client"

The package is not on PyPI yet. Until it is, use the second line, pinned to a tag or a commit. The client pins positronic-evals exactly, and the index supplies it. So a git install of an unreleased client needs that positronic-evals version released first. The platform serves the same contract as an OpenAPI schema at <platform>/openapi.json, and /docs browses it.

platform_client never imports positronic. One command ships here, platform-register, which mints a key through a sign-in. The commands that drive an eval, positronic eval run, eval status, eval list, eval catalog and positronic account, ship with positronic, which depends on this package.

Registering

platform-register mints the API key that every other call needs. It runs a device flow: it prints a short code and a URL, waits while you sign in and approve the code in a browser, and registers the account that signed in. The package install above puts the command on your path.

platform-register --alias=<display name>
export POSITRONIC_PLATFORM_API_KEY=<the key the command printed>

The platform names its identity provider at GET /v1/auth.config, and the command runs the device flow of that provider:

• WorkOS, when the platform names a WorkOS client. You sign in with the methods that the platform's WorkOS environment offers, such as Google, GitHub or an email code. --workos-client-id and POSITRONIC_PLATFORM_WORKOS_CLIENT_ID name another client instead.
• GitHub, when the platform takes GitHub tokens and names no WorkOS client. The command uses the platform's own OAuth app, whose client id it carries as its default. --client-id and POSITRONIC_PLATFORM_GITHUB_CLIENT_ID name another app instead. The GitHub token carries the scopes read:user and user:email.

The command keeps the access token in memory, sends it once to users.register, and keeps no other token. WorkOS also sends a refresh token, which the command drops. The platform reads the account once, mints a key, and stores no token of the provider.

A second run returns the same account and no key: the platform cannot read back a key it issued. Run platform-register --rotate to mint a new key on a machine that lost it.

--platform-url and POSITRONIC_PLATFORM_URL name a platform other than the default. The command refuses a plain http platform that is not loopback. Staging has no TLS and is reached over the tailnet: pass --plaintext-http to reach it. A client that PlatformClient builds itself uses a proxy from the environment only for an https platform, so such a proxy never carries a key or a token over plain http. A client you pass in keeps its own proxy setting.

users.register checks the token with the provider before it answers, and a caller that gives up first loses the key the platform minted. Pass timeout=client.register_timeout() to PlatformClient.register: register_timeout() reads auth.config, where the platform states its deadline for the WorkOS check, and covers the longest check the platform runs. Without timeout, register sends one request and waits 90 s.

Credit accounts

PlatformClient.billing_account(org) reads the member's billing role, credit balance, tariff, and configured purchase packages. One credit is 60_000_000_000 integer units. The account response carries the operator-configured tariff for recorded episodes and duration. The gateway freezes the accepted request's quote and reserves its full maximum before execution. The gateway configures duration_rounding_sec, with a default of 30 seconds, and rounds each episode duration up to that interval. Tariff.for_rates requires an explicit interval. Use zero for exact nanosecond pricing. The quote reserves the charge for the rounded cap. Zero duration adds no duration charge. Tariffs without that field use exact nanosecond pricing and the v1 version hash.

The credit balance belongs to the organization. QuotaLimit values from users.me describe independent limits and use each limit's own scale. The period credits meter uses six units per credit. Its remaining quota does not describe or fund the credit balance. The packaged mode uses the organization's configured invoice terms. The pay_as_you_go mode spends the frozen request quote against its credit balance.

PlatformClient.create_purchase takes a BillingPurchaseCreateRequest with an organization, package id, and transaction key. Reuse the same key to read the same owned purchase after a lost response. A new purchase requires the billing spender role. The configured package fixes its credit units, currency, and amount in currency minor units. currency is the lowercase ISO 4217 code used to verify the Stripe price and payment. For example, amount_minor=17 means 17 yen for jpy, or 0.17 dollars for usd. The currency does not change the package's credit units. The response carries a Checkout URL only when the initiating member can still pay the purchase. Credits appear after the gateway verifies the payment. A browser redirect does not grant credits.

PlatformClient.get_purchase(id) and list_purchases(org) read purchases without creating another Checkout session. list_purchases returns one page with up to 50 purchases by default. The gateway clamps larger limits to 200. Pass response.next as after to read the next page. Continue until response.next is None. The gateway requires a list limit. Older clients that omit it receive HTTP 422 instead of an incomplete history. An organization member can read purchase history. A reviewed or credited purchase carries no payable URL.

positronic account credits account --org=acme
positronic account credits buy --org=acme --package-id=<configured-package> --transaction-key=<stable-key>
positronic account credits get-purchase --id=<purchase-id>
positronic account credits list-purchases --org=acme
positronic account credits list-purchases --org=acme --limit=50 --after='"last-purchase-id"'

users.me lists every organization the caller belongs to in organizations. Older platforms that omit organizations produce an empty list. account, buy and list-purchases use the sole membership when the caller gives no --org. With zero or multiple memberships, they require --org, including when the server omits organizations. An explicit --org selects that organization without fetching memberships. Each command prints the organization it uses on stderr, for example org: acme (sole organization).

These commands print typed JSON with exact integer units and configured currency amounts. Quote a numeric-looking text argument with inner quotes, for example --transaction-key='"20261005"'.

Eval plans

An eval is a list of tasks. The platform offers named evals, and a customer composes one: an EvalPlan names the catalogue tasks to run, the policies (endpoints) to run them on, and the episodes each endpoint takes on each task. A plan either states its own tasks or names an eval the catalogue expands into them; both arrive at the same set. The plan states the count once. A task may override it for that task, and an endpoint may override it for that endpoint, so a 10 + 10 + 2 round is one plan. The scene fields sit on the plan and on a task: tote_placement, camera_vantage, external_cameras, clutter and start_pose. An endpoint states only its count. episodes_total is a checksum a caller may state. max_cap_per_episode_sec is the upper bound on every task's cap.

request_type is required and states the rules a plan runs under. private_eval runs for the org it names, and the caller must be a member of that org. The org's approvals decide the evals, the tasks and the endpoint kinds it may use. nebius_competition names one eval and one image endpoint, and counts against the daily quota. --org states a private_eval on the command line.

rig_shape names the embodiment a plan that states its tasks runs on: franka, a single-arm Franka, or yam, a bimanual YAM. A plan that names none runs on a franka. The platform picks the rig of that shape. A plan that names an eval runs on the embodiment the eval pins, and it is refused when it names a rig_shape other than franka. The gateway refuses sim. The platform runs every task of a plan on the rig its rig_shape names, and it does not check that a task suits that rig: a plan of two-arm tasks states rig_shape: yam.

request_type: {type: private_eval, org: acme}   # or {type: nebius_competition}
rig_shape: franka                        # franka | yam
tasks:
  - eight-spoons-into-grey-tote          # a bare id takes the plan's endpoints and counts
  - task_id: marker-in-mug               # a mapping overrides for that task alone
    episodes_per_endpoint: 2
    cap_per_episode_sec: 120
    endpoints: [candidate]                    # a list replaces the plan's list for this task
endpoints:
  - name: baseline                        # remote: the caller's own server
    wire: websocket_tls                   # names the wire, then that wire's address
    address: {host: baseline.example, port: 443, path: /api/v1/session, query: mode=native}
  - name: candidate                       # served: the platform brings it up and records the address
    kind: served
    spec: dreamzero
    wire: grpc
episodes_per_endpoint: 10
episodes_total: 22
cap_per_episode_sec: 180
max_cap_per_episode_sec: 300
policy_preset: production
tote_placement: random                   # left | right | random | none
external_cameras: {side: random}         # per mount, by the task's name for it
start_pose: droid_reset                  # nominal | droid_reset

start_pose names where the arm starts each episode: nominal, the rig's own pose, or droid_reset, the reset pose of the DROID data collection. A plan that names none takes the pose of each task's catalogue entry. A yam rig opens each episode at its own pose, whatever the plan names.

An endpoint states where its policy comes from (kind) and the wire a session runs over (wire). Every kind names its wire. There is no default.

wire is a name from positronic_wire.registry. Each wire dials its own address fields:

wire	Address fields
websocket	host, port, path, query
websocket_tls	host, port, path, query
websocket_unix	uds, path, query
grpc	host, port, path, query
grpc_tls	host, port, path, query
roboarena	host, port

path is the session route: /api/v1/session. query defaults to empty.

Each kind carries its own locator:

kind	Carries
remote	address, the fields its wire dials
served	spec; the platform brings it up and records the address
image	image, the container; takes only the websocket wire, which the platform opens every image session over

• A record carries no URL. A url field is refused. A scheme in host is refused.
• The kinds refuse each other's fields. An entry carries one answer to each question.

A remote endpoint may state max_concurrent_sessions, an integer of 1 or more. The platform then holds at most that many sessions open to the policy at once. A server that serves one session at a time states 1. With no value, the platform picks the count. Only an entry that names its address states it: a label on a task takes the value of the plan endpoint it names.

Each address field holds one grammar. The wire writes each value into what it dials with no change, so the grammar admits only a value the wire can write as it is:

Field	Holds	Refused
host	A hostname: labels of letters, digits, - and _, joined by ., with an optional last .. Or an IPv4 address, or an IPv6 address with no brackets	A scheme, a port, a path, userinfo, brackets, an IPv6 zone index, any other character. The wire adds the brackets that an IPv6 address needs
port	An integer from 1 to 65535	Any other value
path	The session route, /api/v1/session	Any other route, a checkpoint appended to it included
query	The session params with no leading ?, in visible ASCII characters. Empty by default	A leading ?, a #, a space, any other character
uds	An absolute socket path	A relative path, a NUL byte

Percent-encode a character that query refuses: a space is %20, and # is %23.

positronic eval run --from-file files that plan with submissions.create. The file is YAML or JSON, and an --eval value is a name. Two or more endpoints make one blind sample: the operator is told no policy, and each episode records which one served it. eval status and eval list read it back by the submission id every run carries. The platform records the plan, the rollouts coordinator runs it on a rig of the shape it names, and a blocked run waits on what its reason names. A plan that states its own tasks needs a customer grant; a key without one is refused forbidden, and so is catalog.tasks. Write to hi@phail.ai for a grant. A rig plan queues for an operator, so it answers pending with a queue_position, and it does not count against the submissions.day quota: that quota counts the image runs the platform executes itself. users.me names the grant's client in client.

Each run of a rig plan carries episodes: what the run took on, and what it recorded. done moves as the rig records each episode. A finished rig plan also carries replay, a page that plays back its episodes, and outcome, the kept, judged and successful episodes per endpoint. replay is absent until the platform has built the page. The rig writes the episodes to the org's own bucket, and artifacts.result names the prefix they landed under. Read it with the org's own credential for that bucket: submissions.artifacts refuses a rig plan.

The answer to submissions.create carries resolved, the plan as the rig runs it: episodes_total, and for each task the count per endpoint, the cap, the preset, each side, the vantage, the clutter objects and the episode order. A level states a value, else the task's catalogue entry gives it, else the platform draws it. The platform makes each draw once per plan, so every run of the plan lays out that scene and that table, and runs the episodes in that order. submissions.get carries the same resolved. A rig plan whose task resolves no cap_per_episode_sec or no policy_preset at any level is refused bad_request, and the refusal names each task and what it lacks. Name a preset the rig carries: production serves each episode from one of the plan's endpoints.

submissions.resolve takes the same plan and answers with resolved alone. For a pay_as_you_go organization, that ResolvedPlan carries a credit_quote. It files nothing, spends no quota and returns no submission id. A plan with a transaction_key draws from that key, so a dry run shows the draws a submission under the same key then makes. Without a key, the draws are an example. From Python, PlatformClient.resolve_plan makes the call.

EvalPlan refuses unknown fields. EvalPlan.model_validate(plan) raises on one before anything reaches the platform.

A plan states its own tasks and endpoints. A policy image run names a catalog eval: Submit a policy image says what the platform requires of the image, and how to build, test and submit it.

A policy image is one endpoint of a plan: --policy-image states an image endpoint on the websocket wire, and --eval names the eval whose tasks it runs. plan_of_image builds that shape.

A plan that names an eval may run a random sample of the eval's trials. sample states the size as count, a number of trials, or as percent, a share of the eval's trials rounded up, and a seed (0 when none is given). The same seed draws the same trials, and the run takes them in the eval's own order. A count larger than the eval's trials is refused bad_request. A sampled run ranks on no board. A plan that states its own tasks takes no sample. On the command line, --sample-count or --sample-percent states the size and --sample-seed the seed.

eval: molmo.franka_pick_mini_full
sample: {percent: 10, seed: 7}           # or {count: 50, seed: 7}

An image endpoint whose registry serves no anonymous caller states image_credential. A plan file names the registry user and the FILE the password is in. positronic eval run --from-file reads it as EvalPlan[RegistryCredentialFile], then plan_with_passwords_read gives the EvalPlan a request carries, whose RegistryCredential holds the password. From Python, credential_from_file builds that credential.

endpoints:
  - name: policy
    kind: image
    wire: websocket
    image: registry.example.com/you/policy@sha256:...
    image_credential:
      username: a-reader
      password_file: ~/.config/positronic/registry-password

positronic eval catalog prints what the key may name: catalog.evals lists the evals a plan names, and catalog.tasks the tasks a plan may compose, each with its embodiment. Every registered user sees the evals a submission can name. A customer grant adds the rig's evals and tasks, filtered to the entries offered to the grant's client.

From Python, PlatformClient takes and answers the models in platform_client.eval_plan and platform_client.catalog. The rollouts coordinator's request record is a subclass of EvalPlan, so the ask has one definition.

ResolvedPlan.rig_shape names the plan's rig shape. Missing or invalid shapes fail response validation.

Public evals, labels and scores

positronic-evals holds the definitions of the public evals, the labelers and the scoring rules. Its README describes them. An eval name, EvalRef, is positronic_evals.evals.EvalRef.

From the command line

positronic carries the other commands, and a checkout needs no installation step. eval run runs an eval here when given a policy, on the platform when given a policy image, and on the lab rig when given a plan file. To get a key, run platform-register, which prints the export line, or account register, which saves the key in the record the other commands read. Both run the device flow of the provider that the platform names. In a checkout, run them through uv run. With a token in POSITRONIC_PLATFORM_CREDENTIAL, account register registers with that token and runs no sign-in. Its sign-in refuses a plain http platform that is not loopback unless you pass --plaintext_http. A token from POSITRONIC_PLATFORM_CREDENTIAL goes to the platform as given, like the key of every other command.

platform-register --alias=<display name>            # in a checkout: uv run platform-register
export POSITRONIC_PLATFORM_API_KEY=<the key it printed>

uv run positronic eval run --eval=<name> --policy-image=org/policy@sha256:…
uv run positronic eval run --from-file=positronic/cli/examples/rig_plan.yaml --org=<org>
uv run positronic eval run --from-file=positronic/cli/examples/yam_plan.yaml --org=<org>
uv run positronic eval status --id=<hex id>
uv run positronic eval list
uv run positronic eval cancel --id=<hex id>
uv run positronic eval catalog

positronic/cli/examples/ runs the whole flow end to end.

A command that fails exits with a status that tells whose fault the failure is:

Status	Fault	Cases
1	the caller	the platform refused the request with a 4xx, no API key or a bad key, a quota used up, a run rejected at the door, a sign-in the provider refused (an unknown client id, a refused or expired code)
2	usage	the arguments are wrong before any call: an id that is not hexadecimal, a plan file that does not parse
3	the platform	a 5xx, an internal_error or upstream_unavailable code, no answer, a timeout, an answer the client cannot read, a replayed run that ended for a platform reason, an identity provider that is unreachable or unavailable, a platform that takes no sign-in this client can run

The read commands take --json: eval status, eval list, eval catalog, and account credits account, get-purchase and list-purchases. They print the data as JSON on stdout. eval list --json prints every page as one page, with next null. eval catalog --json prints the evals and the tasks as one object, with tasks null where the key may compose no plan. A refusal by the platform prints its error envelope as JSON on stdout. Any other failure prints its message on stderr. The exit status is the same with and without --json.

An argument that a command does not declare, or a required argument that is missing, stops with a traceback and status 1. --help lists the arguments of a command.

Configuration

Calls go to https://platform.positronic.ro with nothing set. The environment carries the rest:

Variable	Holds
POSITRONIC_PLATFORM_URL	a platform other than the default one, overridden per call by --platform-url
POSITRONIC_PLATFORM_API_KEY	the key register mints — read from the environment or the saved record, never an argument, so it reaches no process listing
POSITRONIC_PLATFORM_CONFIG_DIR	where positronic account register saves that record, else ~/.config/positronic-platform
POSITRONIC_PLATFORM_CREDENTIAL	a token of the platform's identity provider, for account register in a script — read the same way, for the same reason. Without it, account register signs in itself
POSITRONIC_PLATFORM_WORKOS_CLIENT_ID	a WorkOS client other than the one the platform names, for both register commands
POSITRONIC_PLATFORM_GITHUB_CLIENT_ID	a GitHub OAuth app other than the platform's own, for both register commands

The record's key and its platform are read one at a time. A command that names another platform, and no key, sends the record's key to the platform it names. The client speaks one wire contract and runs against any platform that serves it, but it is built for ours. You take the risk of another one.

rankings.list names the boards, and rankings.get?board=<slug> reads one. From the command line, positronic/cli/examples/nebius_competition/standings.py prints the boards, or the rows of one:

uv run positronic/cli/examples/nebius_competition/standings.py
uv run positronic/cli/examples/nebius_competition/standings.py --board=<slug>

From Python, PlatformClient.list_boards and .rankings read them, the latter taking a BoardRef (platform_client.boards). Both take the key when one is set and work without: a public board is readable by anyone, a tenant's board only by its members.

A board row reads <display name>#<tag>. The name is an alias and is not unique — a board may hide it altogether — so the tag is what tells two rows apart, and it is how you find your own: it is the same on every board you appear on.

A row with baseline set is a reference model, and baseline.publisher names the organization that released it. A baseline row has no rank. The other rows rank 1..N among themselves, and the board lists every row in score order.

Each row carries replay, a page that plays back the camera video of the row's submission, with the task and outcome of each episode. The page opens without a key, and replay is absent until the platform has built it.

replay.data_url names the JSON object the page renders from. about identifies the submission, and each episode carries its values and one absolute URL per camera video. A page on another origin can read the object, the videos and both rankings routes with no key:

{
  "title": "<board title> — 0ddba7",
  "url": "https://app.positronic.ro/v/<token>/",
  "about": [
    {"key": "board", "label": "Board", "value": "<slug>"},
    {"key": "eval", "label": "Eval", "value": "<eval>"},
    {"key": "tag", "label": "Tag", "value": "0ddba7"},
    {"key": "submission_id", "label": "Submission", "value": "<hex id>"},
    {"key": "primary_score", "label": "success_rate", "value": 0.35}
  ],
  "columns": [{"key": "task", "label": "Task"}, {"key": "outcome", "label": "Outcome"}],
  "episodes": [
    {
      "index": 0,
      "values": {"task": "<task>", "outcome": "Success"},
      "duration_s": 24.1,
      "videos": [
        {
          "camera": "image.exterior",
          "url": "https://app.positronic.ro/v/<token>/episodes/0/image.exterior.mp4",
          "src": "episodes/0/image.exterior.mp4",
          "width": 640,
          "height": 360,
          "duration_s": 24.1
        }
      ]
    }
  ]
}

columns labels the keys of each episode's values. src is the video's path relative to the object. Every video of an episode starts at the episode's first camera frame, so one clock plays them together.

From Python

from platform_client.client import PlatformClient
from platform_client.eval_plan import plan_of_image
from platform_client.policy_images import PolicyImage

with PlatformClient(api_key=key) as client:
    boards = client.list_boards().boards  # public; each names the eval it ranks
    for board in boards:
        print(board.board, board.eval, board.primary_metric)
    created = client.create_submission(plan_of_image(PolicyImage('org/policy:v1'), boards[0].eval))
    view = client.get_submission(created.submission_id)

The platform resolves the image reference to a digest at submission and records it as policy_image_digest, so a tag is safe for the run: the run uses the bytes the tag named at that moment. Pin by digest to know which image was scored.

An eval is the whole of what a submission chooses: it names a task suite and the embodiment that runs it — one simulator, or one real robot — so there is no second axis to get wrong. The platform owns the set of names, and PUBLIC_EVALS names only the public ones. Do not copy a name from a document; read it from the platform:

• rankings.list needs no key. Each board it returns names the eval it ranks.
• submissions.create with a name the platform does not offer raises a PlatformError whose evals carries every eval on offer, with a board or without one.

Ids are 64-bit ints in Python and bare lowercase hex on the wire; closed sets are IntEnums carried as slugs. A non-2xx response raises PlatformError, which carries the parsed error envelope: code for a program, message for a human, reason_code where a terminal caller fault has one, quota where a 429 names the rule that refused the request, and evals where the eval asked for is not one of them. A response with no envelope, such as a proxy's HTML page, still raises PlatformError: the client builds an internal_error envelope around the raw body and sets synthesized_envelope. ReasonCode.is_platform_fault says whether a reason is the platform's fault, which refunds the quota, or the caller's, which charges it.

submissions.get answers one model per status. A finished run reads its score off scores and its outputs off artifacts. A failed run reads reason_code and reason, and artifacts links the records the run wrote: result is its outcome, and diagnostics says why its policy failed. policy_log is the policy container's own output, and a finished or failed run carries it. Each link is a signed URL, because the submitter holds no credential for the bucket. artifacts is absent on a run that wrote nothing, diagnostics on a run whose record was not written, and policy_log on a run whose container printed nothing.

submissions.artifacts lists what a finished run wrote, one page at a time. Each entry names its key under the submission's own prefix, its size, and a signed url that expires. Pass prefix to keep the page to one part of the tree, so one attempt's episodes are attempts/<n>/episodes/ and result.json's attempt_location names the attempt that scored. Pass after with the next of the page before it to read the rest. The route answers a finished run alone: a run still going has a part-written prefix, and a run that failed reads its records off submissions.get. It refuses a rig plan, whose episodes are in the org's own bucket (§Eval plans).

users.me reports the plan's rules as a list of QuotaLimit, each with its own key, window and subject; MeResponse.quota_for(QUOTA_SUBMISSIONS_DAY) reads one by key, from the keys the package publishes beside QuotaLimit.