positronic-model-server

Serve a native model inside its own Python environment, without Positronic, Torch or JAX dependencies. The wrapper owns transport, sessions, serialization and cleanup. Vendor scripts load and call their model directly in the same process and describe the client pipeline as plain data.

uv pip install "positronic-model-server==0.2.2"
uv pip install "positronic-model-server[websocket]==0.2.2"
uv pip install "positronic-model-server[grpc]==0.2.2"

The core depends on NumPy, Pillow, msgpack and positronic-wire's dependency-free transport definitions. The extras select transport libraries independently; combine them as [websocket,grpc] when both are needed. WebSocket serving uses Uvicorn directly; gRPC does not require it. Starlette, AnyIO, FastAPI and Pydantic are not dependencies of either installation. In a checkout, install the workspace packages together: uv pip install ./wire ./model_server.

Serving and lifetime

This test server returns each observation as a one-element result list:

from positronic_model_server.server import Model, ModelServer, Session
from positronic_model_server.server_wire import ServedHostPort
from positronic_model_server.spec import component
from positronic_model_server.websocket_wire import WebsocketWire

class EchoSession(Session):
    def __init__(self, fps):
        super().__init__(component("chunked_schedule", version=2, fps=fps))

    def infer(self, observation):
        return [observation]


class EchoModel(Model):
    def __init__(self):
        super().__init__(parameters={"fps": 20}, metadata={"checkpoint_id": "example"})

    def prepare_session(self, params):
        if params["fps"] <= 0:
            raise ValueError("fps must be positive")
        return EchoSession(params["fps"])


server = ModelServer(EchoModel, idle_timeout_min=10)
server.serve([WebsocketWire(ServedHostPort("0.0.0.0", 8000))])

Implement Model.prepare_session and Session.infer in subclasses. Each Session owns its state and may reference the loaded model. Override close() to release owned resources; its default does nothing. The constructors accept the client description, parameters, metadata and output image settings.

Each vendor supplies its own script and argument parsing. The factory passed to ModelServer can be a model class or a function. It loads weights and finishes shared warm-up before returning. Listeners bind afterwards, so any successful keepalive answer means shared startup has finished. The factory owns cleanup if loading raises.

Model.parameters declares accepted session names and defaults. Query values are decoded as JSON, or retained as strings when they are not JSON. Duplicate or unknown names are refused; strings never name Python imports. Every session gets an independent copy of the resolved parameters. prepare_session decides how they affect inference and the client description, validates their meaning and completes configuration-specific warm-up. It returns a Session only when the first real inference needs no deferred loading or compilation. Warm-up must preserve the intended initial session state. A preparation that raises owns cleanup of anything it acquired.

The server sends waiting messages during preparation, including while another session holds the model. It then sends ready with a session ID, protocol v3, the pipeline description and metadata:

• session_params: the received query parameters, after decoding;
• effective_params: those parameters with declared defaults applied;
• model_server_version: this distribution's version;
• model/session metadata supplied by the vendor, plus the listener address.

The client may report any additional metadata and owns how it records or combines that information. The server checks description structure without importing client components.

All model operations run on one worker thread in the server process: load, preparation, inference, session cleanup and model cleanup. Each session retains its own settings and history. Inference is serialized across all listeners. Queue time is reported separately from model time. The event loop answers keepalive while the worker is busy; no waiting messages interrupt an inference response. A failed inference reports an error and keeps the session open. A malformed session request ends that session.

Clients own recovery from reported inference errors. The recommended client behavior is to close the affected session and, if retrying, open a fresh session and wait for ready. Continuing in the same session is an explicit client choice for failures it knows are recoverable. An open session does not guarantee that a failed inference left the model's state valid.

Session.close runs after outstanding work, on explicit close, disconnect or server shutdown. The end-session acknowledgement follows cleanup. Model.close runs after sessions finish and listeners stop, including when startup fails after loading. Call server.shutdown() from any thread; an idle timeout ends the server only when no sessions remain. A model operation must return for graceful shutdown to complete.

Model-specific implementations own cleanup and its failure policy. Cleanup methods are expected to release their resources and complete without raising. If Session.close raises, the wrapper logs the failure, attempts to report it to the client and ends the connection; it continues serving other sessions. A cleanup failure can leak resources or leave shared model state invalid. The implementation is responsible for recovery or for requesting server.shutdown() when continued serving is unsafe.

Pass auth_token to require a bearer token on both session and keepalive calls. None serves open; empty or malformed tokens are refused at construction. For gRPC, use grpc_wire.GrpcWire with its own ServedHostPort. Both listeners can be passed to one server and share its loaded model.

Protocol compatibility

The shared server sends v3 results without interpreting robot-command fields or reshaping chunks. Updated Positronic clients select this behavior while preserving v1/v2 command decoding for legacy servers. A client that only supports v1/v2 rejects v3 during opening. Upgrade clients before migrating their endpoint. Legacy servers and their server-side codecs remain supported in positronic.offboard.

Client pipeline descriptions

Vendor code describes components as JSON-compatible data. Only the Positronic client imports and instantiates their implementations:

from positronic_model_server.spec import component, parallel, sequence

description = sequence(
    component("chunked_schedule", version=2, fps=20),
    parallel(
        component("restrict_image_size", width=320, height=180),
        component("metadata", values={"model": "example"}),
    ),
)

component accepts a name, explicit version and keyword arguments, normalizing arguments through JSON. sequence and parallel reject empty compositions. Component names and argument meanings belong to the client registry; these helpers do not resolve classes or validate model parameters. The structural keys (NAME, VERSION, ARGS, SEQ, PAR) have one definition in spec.

The wrapper defines only constants used by its own production code. Vendor settings belong to small vendor modules that training and serving can both import; dataset file conventions belong to the Positronic exporters and their producers. Client component names belong to the client implementations; vendor descriptions spell those names as part of the client's wire contract. Validate each description against the real client registry in vendor tests. The wrapper forwards model metadata and parameter values without owning their field names.

Serialization

from positronic_model_server.serialization import deserialise, encode_jpeg, serialise

message = serialise({"camera": encode_jpeg(image, quality=90), "state": state})
restored = deserialise(message)

Plain values, nested mappings/sequences, NumPy arrays and NumPy scalars use the shared msgpack representation. Ordinary arrays are lossless, including arrays whose dimensions resemble images. JPEG compression is explicit: call encode_jpeg on the values to compress. It accepts RGB arrays shaped (..., H, W, 3), casts pixels to uint8 and preserves every leading dimension. It is lossy. The receiver recognizes each encoded value without a field schema. The same functions encode observations or model results, including returned images.

serialization.JpegEncoding(path, quality) selects a nested mapping/list value, with an empty path selecting the whole result. Supply these selectors through Session.output_images for returned images. serialization.encode_images applies the same selections to observations without changing the source containers. A missing path fails instead of silently skipping compression.

The client component encode_images defaults to recursively JPEG-encoding uint8 arrays shaped (..., H, W, 3) with positive height and width, including batches, inside mappings, lists and tuples. Supply paths to encode only selected values; paths=[] disables compression. quality sets the JPEG quality. Automatic detection is a heuristic: float images are left lossless, and unrelated uint8 arrays matching the image shape are compressed. Its result decoding leaves native values unchanged. The wire serializer itself does not infer image fields.

Nonempty 3D and 4D images retain the v1/v2 JPEG representation. Extra leading dimensions and empty batches carry a full-shape marker; send those only to receivers supporting it. NumPy object, structured and complex dtypes are unsupported. Python tuples arrive as lists.

protocol owns shared message fields, authentication, timing and protocol version names. Serialization here never interprets robot-command fields or __cmd__ envelopes. Positronic's legacy offboard protocol owns those hooks and its supported-version selection.

Package checks

Run uv run pytest model_server positronic/offboard/tests positronic/policy/tests wire from the repository root. CI also builds wheels and installs them in isolated environments with NumPy 1.26.4 on Python 3.11 and 3.12, selecting no transport, WebSocket alone, or gRPC alone. Root tests exercise the repository's locked NumPy version.

Runtime dependencies are a design constraint. Before installing test tools, CI checks every installed distribution against the per-transport allowlist in utilities/check_model_server_package.py. Any additional runtime package fails the check, whether direct or indirect, including dependencies introduced by upstream package updates. Transport dependencies must stay optional. Changes to the allowlist require explicit design review; do not expand it merely to make CI pass. The dependency check uses only the standard library.

The same check scans every wrapper Python module for module-level constants with no internal use. Adding one fails CI, including in a new module. Test/example references and imports or assignments that only re-export a constant do not count. Move such definitions to their owning vendor, client or legacy protocol module; do not add artificial uses to satisfy the check. Source references establish use, while review checks that the behavior itself belongs to the wrapper.