adaptive-serve 0.1.0

Adaptive cloud-to-robot serving layer that scales robotics model quality to fit device, power, latency, and bandwidth constraints.

adaptive-serve

Adaptive cloud-to-robot serving layer for robotics models of all types.

It takes any trained robotics model and serves it to a robot, scaling model quality up or down to fit the binding constraint: on-robot compute (TOPS/VRAM), power budget, latency tolerance, and live network bandwidth. Think adaptive bitrate streaming, but the renditions are model variants and the network is the robot's compute and link.

The whole core runs with no GPU, no network, and no external dependencies. An optional torch backend and a stdlib HTTP cloud adapter add real execution when you want it.

Why

A robot cannot round-trip to the cloud for every control step, and it must keep acting if the link drops. adaptive-serve picks the right variant and serving mode for the device in front of it, runs a slow cloud planner and a fast local reflex loop at the same time, and degrades safely when the cloud goes away.

Install

pip install -e ".[dev]"        # core plus test tooling
pip install -e ".[dev,torch]"  # also pull torch for the real backend

Try it

adaptive-serve demo                                  # full end to end demo
adaptive-serve serve --model planner --vram 2 --bandwidth 200 --difficulty hard
adaptive-serve serve --model planner --vram 80 --bandwidth 0  --difficulty hard

The same model adapts to the device, exactly like adaptive bitrate:

device               mode    variant                 ~latency ms  meets
tiny-edge-offline    local   planner-vlm:2B:int8            36.0  False
edge-weak-link       split   planner-vlm:7B:fp16           761.8   True
edge-fast-link       cloud   planner-vlm:7B:fp16            21.0   True
workstation          local   planner-vlm:7B:fp16             4.2   True

And it keeps acting through a cloud outage, degrading from NOMINAL to a latched emergency stop only when the last plan is long gone:

Multi-rate session with a cloud drop at 1000 ms:
  reflex steps: 1200
  plans published: 2
  safety mode timeline: {'nominal': 200, 'degraded': 531, 'hold': 400, 'kill': 69}
  final safety mode: kill

Calibration: decide on real numbers, not guesses

By default the router uses portable heuristics for footprint and latency. To ground its decisions in real hardware, measure actual models and route on those numbers instead:

adaptive-serve calibrate            # measure real models -> calibration.json
adaptive-serve pipeline             # calibrate -> route on measured -> run in torch

calibrate records the exact parameter count, the weight memory at each precision, and the measured forward latency per variant. A ModelRegistry built with that table feeds the router measured footprints (heuristics remain the fallback for any variant you have not measured), so a model that is actually small gets served where the heuristic would have offloaded it.

Honest scope: weight bytes and params are device-independent and drive the VRAM fit directly; latency is device-specific, so it stays informational. On a CPU box the harness measures real but small reference models and INT8 via torch dynamic quantization; the 7B/70B rungs and true INT4 need target hardware, which the same harness supports. Requires the torch extra: pip install -e ".[dev,torch]".

Supported model classes

The registry and router are generic across these, never hardcoded to one:

• VLA / action policies (diffusion policies, ACT, flow-matching)
• world / dynamics models (Dreamer-style, latent predictors, video predictors)
• perception (detection, segmentation, depth, optical flow, 6DoF pose, keypoints)
• VLMs / high-level reasoners (planning, language grounding)
• state estimation / SLAM / odometry
• classical or learned planners and controllers

Each declares its role in the stack, native rate (Hz), latency budget, and which scaling techniques it supports. Adding a new class touches no core.

Architecture

DeviceProfile  ---+
                  |
ModelSpec --> Registry: variant ladder (size x quant) per model, any class
                  |
                  v
             Router: pick variant + quant + serving mode (local / cloud / split)
                  |
   +--------------+--------------+
   |                             |
Scaling strategies          Orchestrator: cloud System-2 planner ~1 Hz
size/quant/input/exit/      + local System-1 reflex 50-1000 Hz
cascade/split                    |
   |                             v
   +--------------->  Safety governor: hold / kill on loss
                             |
                             v
                      Runtime + backends: run local (stub or torch) or
                      offload over HTTP, fall back to local on failure
                             |
                             v
                      Telemetry: record every step

Two invariants hold throughout, both covered by tests:

• Generic across model classes. The router and scaling cores dispatch on a spec's declared role, tier, latency budget, and supported techniques, never on class identity.
• The robot never blocks on the cloud. The reflex tier runs every control tick. On cloud loss the planner stops publishing, the reflex keeps acting on its last plan, the safety governor degrades to HOLD and latches KILL when the plan is too stale, and a late or failed cloud call is dropped for a safe local action rather than awaited.

Develop

ruff check .
pytest

See STATUS.md for the full component list and PLAN.md / ADAPTERS_PLAN.md for the build history. MIT licensed.