waddle-sdk (Python)

The Python frontend owns one customer site: strict configuration, hardware and camera lifecycle, the owner envelope, paired timestamps, and raw recordings. Applications consume the structural waddle_sdk.runtime.SdkRuntimePort; the SDK never imports or discovers application implementations.

Site lifecycle

import waddle_sdk

site = waddle_sdk.load_site("site.yaml")
with site.open(transport=waddle_sdk.Grpc(url, token)) as session:
    with session.run(task={"id": "pick"}, actor={"id": "controller"}) as run:
        observation = run.observe()
        result = run.step(action, observation)
        if not result.dispatched:
            run.hold(result.detail or "command withheld")
    session.estop("emergency")

Exiting a Run without an explicit terminal result records ABORT. Exiting the site context finalizes the recording and closes cameras, pumps, robot handles, and native threads even when opening or the body failed. Run.step asks the native gate first, then preflights every addressed owner-envelope target before writing any part; one refusal holds the addressed set and moves none of it. The public lifecycle exposes no lease or handoff selector: handoff is fixed to hold-first and enforcement placement is derived from the selected integration.

The root package deliberately exports only Site, SiteSession, Run, load_site, Grpc, LiveKit, Outcome, manifest errors, ConnectorRegistrationError, and ConnectorCompatibilityWarning. Driver-extension contracts live in their subpackages. The older module-global lifecycle, local web UI, hosted task/artifact facades, and execution-backend discovery are not part of the public surface.

Portable Agent Skills

Each wheel includes waddle-sdk-contracts and port-waddle-hardware for coding agents working against that installed SDK version:

waddle-sdk skills list
waddle-sdk skills list --json
waddle-sdk skills export port-waddle-hardware --output ./agent-skills

Export creates a new complete skill folder beneath the chosen directory and refuses an existing target. It does not install or enable the skill and never edits global agent configuration. Interpreter-generated bytecode and cache directories are excluded, so the result contains only the packaged, version-matched resources. The porting skill scaffolds and validates non-opening adapter code; it contains no live-motion or automatic commissioning command.

Hardware discovery for configuration frontends

waddle_sdk.discovery.discover_hardware() performs a non-opening scan for Linux CAN/serial transports and camera devices. It returns immutable evidence; it never constructs a driver, opens a robot bus, starts a camera pipeline, or guesses that a generic CAN device is a particular robot. Configuration tools must ask the site operator to choose the adapter and provide owner-envelope measurements that cannot be discovered safely.

Custom adapters may publish a callable in the waddle_sdk.hardware_discovery entry-point group. The callable returns HardwareCandidate rows and is isolated from other providers if its optional vendor probe fails. This extension API intentionally remains in its subpackage and is not added to the small SDK root exports.

Identifying several camera views is an explicit second step. Build CameraInspectionSpec objects from exact camera candidates, then enter waddle_sdk.cameras.inspect_cameras(specs). The returned context opens only those cameras, retains one latest immutable frame per camera, and closes each driver before joining its capture thread. It never opens a robot part or creates transport, control, media-publication, or recording state.

site.yaml

api_version: waddle.site/v1
kind: Site
metadata: {id: customer-cell-1}
parts:
  left:
    driver: waddle_sdk.robots.yam:arm
    posture: supervised
    base_frame: yam_left_base
    connection:
      channel: can_left
      configure_can: true
      can_bitrate: 1000000
    joint_limits: {}
    gripper:
      joint: gripper
      closed_m: 0.0
      open_m: 0.095
      closed_action: 0.0
      open_action: 1.0
      closing_axis_tcp: [0.0, 1.0, 0.0]
      pinch_offset_tcp_m: [0.044, 0.0, -0.0049]
      pointing_down_wxyz: [0.0, 0.0, 1.0, 0.0]
    options:
      gripper_limits: [1.7, 0.1]
      gravity_comp_factor: [1.0, 1.1, 1.2, 1.3, 1.0, 1.0]
      arm_gain_scale: 1.0
      gripper_gain_scale: 1.0
      velocity_feedforward: true
      max_feedforward_vel_rad_s: 3.0
cameras:
  overhead:
    driver: waddle_sdk.cameras.realsense
    mount: {kind: scene}
    connection: {serial: "207322250310"}
    stream: {width: 1280, height: 720, fps: 30}
frames: {}
calibration: {artifacts: calib/}
workspace_bounds: {}
envelope:
  static_keepouts: []
  self_collision: {}
recording: {root: data/, format: mcap}

The Draft 2020-12 schema rejects unknown fields. Site paths are relative, portable, normalized, and confined beneath the manifest directory. Values with credential-like names must be {secret: NAME} and are resolved only while the site is being opened. A site manifest cannot contain graphs, skills, models, chat configuration, API keys, lease modes, or application workspace paths.

parts.*.base_frame names the coordinate frame in which that opened arm reports poses. Built-in adapters consume it directly, and the site lifecycle rejects a custom driver whose opened arm reports a different frame. cameras.*.mount is physical topology, not a transform: {kind: scene} means fixed to the site, while {kind: wrist, part: left} means the camera moves with that exact part. Higher layers use this declaration to decide which calibration artifact and paired robot observation are required; the SDK does not invent or solve the transform.

For YAM, connection.configure_can: true asks the SDK to inspect only the named SocketCAN interface before I2RT opens it. If that link is down, the SDK configures connection.can_bitrate and brings it up, using the process's network capability or the same bounded ip argv through sudo. An already-up link is never reconfigured: a bitrate mismatch fails with the actual and expected values. Omit or set configure_can: false when systemd-networkd, NetworkManager, or another site-owned service manages the bus. Custom robot adapters can reuse waddle_sdk.robots.ensure_socketcan_up; the helper never scans for other links or guesses what is attached.

Before a live YAM opens I2RT, its reference adapter installs an exact-signature workaround for the pinned vendor receive loop. The stock loop polls SocketCAN in one-millisecond slices and can abandon a healthy reply if Python is descheduled across its wall-clock deadline, leaving that reply to poison the next motor transaction. The adapter instead makes one kernel wait plus a final non-blocking poll for each receive deadline. Transactions accept the expected reply throughout the existing 10 ms initial and 9 ms recovery windows, including a valid late reply that the stock recovery path discards. Unrelated reply IDs do not restart those deadlines. Explicit reply IDs for passive encoders, matching motor-error frames, command bytes, retry counts and communication-error text are preserved. Both vendor method signatures must match before installation; repeated arm opens retain installed patches and marked instrumentation. These workarounds are local to the YAM adapter; custom drivers are unchanged. They do not establish a real-time scheduling guarantee or cure physical CAN, USB or motor faults.

A simulator that owns one shared scene, clock, and renderer declares a worlds entry. Its parts and cameras name world: <name> instead of separate device drivers. The SDK opens the selected backend once after connector authorization, advances it once per composite tick, and closes it after its cameras and arms. World-backed devices still produce the same runtime description, support rows, observations, actions, and RGB-D samples as physical devices. The manifest and public extension contract are documented in Simulation backends.

Reference simulation parts may explicitly declare max_joint_position_error_rad in their options to match a hardware site's scalar or ordered arm-joint bounds. Omission preserves the existing speed/rate fallback. This uses ordinary SDK admission and support facts; it does not change motor gains or certify dynamics.

For a URDF-based environment, use the portable scene build instead of hand-authoring simulator XML:

waddle-sdk sim backends
waddle-sdk sim init arm.urdf --tool-link tool0 --output portable-scene
waddle-sdk sim validate portable-scene/scene.yaml
waddle-sdk sim compile portable-scene/scene.yaml \
  --backend mujoco --output portable-scene/build

waddle.scene/v1 declares robot placement and reviewed limits, scene/wrist camera optical poses, RGB-D streams, lights, material/link overrides, visual or physical coating geometry, fixed geometry, free rigid bodies with explicit inertia, and seeded variation. Named parent-child bodies can add passive bounded slide or hinge fixture joints with damping, friction, and optional springs. Compilation is atomic and emits an ordinary site.yaml, a native world, normalized inputs, and resolved hash evidence. Optional complete scene/embodiment identity reaches only the generated evidence and retained simulation-administration snapshot; ordinary participant runtime descriptions remain backend-neutral. Physical coatings require an explicit conservative safety sphere. Installed simulator packages can register short names under waddle_sdk.simulation_backends and waddle_sdk.simulation_compilers; applications need no simulator-specific runtime.

The shipped ros2 world backend connects Gazebo, Isaac Sim, or another ROS graph. It consumes named joint state, RGB, depth, and CameraInfo topics and publishes either ros2_control Float64MultiArray position commands or Isaac-compatible JointState commands. ROS 2 is supplied by a sourced ROS installation and is imported only when that world opens.

parts.*.gripper is driver-neutral control metadata: it maps a physical jaw opening in metres onto one declared action row. It is visible through Site.describe() but is never forwarded into the driver factory. Adapter construction facts such as a YAM unit's measured motor limits remain under options. A site that enables generic Cartesian grasping declares all three optional grasp-geometry fields together: the unit TCP-frame closing axis, the TCP-to-pad-contact offset in metres, and the canonical pointing-down quaternion in wire-order wxyz. Omitting the set leaves joint/gripper control available without claiming that a higher layer knows how this hand grasps. The YAM values above preserve the geometry and quaternion conventions derived from its reviewed model; another hand must publish its own facts rather than copying them.

YAM limits retain I2RT's semantic [closed, open] order; depending on that unit's motor direction, the measured pair may be descending and must not be sorted. The gripper_limits option is an override, not a required measurement: omit it to let I2RT perform its normal jaw auto-range when the live hardware opens. That calibration moves the jaws at connection time; supplying a measured pair skips it. The driver-neutral gripper record remains independent and describes normalized actions and physical jaw geometry to higher layers. gravity_comp_factor sets the six absolute arm gravity-torque factors, in joint order. The SDK defaults are [1.0, 1.1, 1.2, 1.3, 1.0, 1.0]: joints 3 and 4 use rounded two-arm bench calibration; the remaining rows retain the pinned vendor values. These are control defaults, not universal per-unit calibration. For a specific arm, put its six finite positive values in parts.<part>.options; the vector replaces these defaults rather than multiplying them. I2RT receives the vector before its servo starts and appends its unchanged gripper factor of 1.0. Changing configuration takes effect on the next hardware open. The kinematic sim: true driver does not model gravity and is unaffected.

arm_gains optionally supplies separate six-element kp and kd vectors under parts.<part>.options, so position stiffness can change independently of damping. It is mutually exclusive with arm_gain_scale != 1; that legacy scale multiplies both gains for all six arm joints. gripper_gain_scale affects only the seventh motor. Defaults remain the vendor's gains. Effective values must be finite and positive, with KP at most 500 and KD at most 5. Configuration is checked before CAN starts; a request the vendor would silently clamp now fails. In particular, default arm KD is already 5 on joints 1–3, so increasing arm_gain_scale is refused. These are encoding limits, not recommended tuning values. Requested gains survive e-stop recovery; simulation and monitor modes validate but do not apply PD gains. See YAM gain configuration. When an application supplies a trajectory's known joint velocity, the YAM adapter uses I2RT command_joint_state for simultaneous position/velocity control. It never differentiates measurements or an IK stream to invent velocity, always sets the gripper's velocity to zero, and degrades to command_joint_pos on an I2RT build without that method. velocity_feedforward: false selects the same position-only degradation explicitly; max_feedforward_vel_rad_s bounds the motor hint independently of the declared motion speed. For the pinned I2RT revision, the adapter constructs the complete position/velocity/PD command before swapping it under I2RT's command lock; its server loop therefore always adds gravity and friction compensation to one complete active command.

static_keepouts accepts strict axis-aligned box and sphere records with an ID, collision frame, optional part filter, and non-negative margin. self_collision enables deterministic named-body checks within and across parts, with a shared margin and explicit ignore_pairs for adjacent bodies. The selected driver adapter supplies conservative base.CollisionSphere geometry in one collision_frame; all intersection policy and reject-whole dispatch remain in the SDK. A configured rule with missing geometry or a frame mismatch fails closed while the arms are opened.

Runtime port and remote admission

SdkRuntimePort is the same contract for direct-library and transport adapters: describe, begin_run, observe, submit, hold, estop, cursor-based events, and calibration measurements. Its DTOs and structured faults are in waddle_sdk.runtime. Authority and time stay native-core owned. JointPositionCommand adds an optional known velocity feedforward to a position action. Drivers without the optional base.PositionVelocityDriver extension receive the position normally, so the contract does not make velocity support a prerequisite for motion.

An opened local SiteSession also implements three additive, structural facets: SdkSupportPort, SdkKinematicsPort, and SdkGeometryPort. The support port returns an immutable SupportMatrix and describe() publishes the same data under support using waddle.sdk.support/v1. Its rows report SDK support facts, not robot skill capabilities: applications intersect them with the exact registered action space and grants before enabling a skill. A hardware-specific FK or body-geometry implementation can therefore refine one facet without manufacturing motion permission or making unrelated tools fail. The matrix digest identifies the complete public site embodiment, while every row has its own scope digest: a robot row binds only that part's action space, declared base frame, portable model/action-frame ancestry, and gripper declaration; a camera row binds only that camera's public declaration. Grants, live status, connection data, and unit/site identity are excluded, so changing an unrelated camera cannot invalidate a hardware-specific robot implementation match. Existing vendor modules need no extra adapter surface: Arm.fk and Arm.collision_spheres back the optional runtime facets automatically. Missing facets remain honestly absent so applications can select a generic implementation whose declared prerequisites are present or mark only the dependent skill unavailable. Camera rows intentionally do not advertise aligned depth until a stable depth declaration exists; observing one transient RGB-D sample is not a persistent hardware fact.

Remote waddle.v0 sessions negotiate waddle.v0.motion.feedforward; accepted peers preserve the hint through the core gate and raw recording, while older peers ignore it and execute the identical position target. This is a generic SDK capability: from waddle_sdk.robots import PositionVelocityDriver is the public structural extension point for any customer or vendor adapter.

A connector first advertises waddle.v0.connector.binding in an authorization_only Register carrying the exact customer, project, and workspace. The host must authenticate that tuple and accept the flag before SiteSession invokes any hardware builder; a refusal, old host, or timeout opens nothing. The probe does not advertise hosted runs. The runnable connection then registers the same tuple again. Every gRPC method carries that exact binding and one fresh per-connection nonce as non-secret metadata; Register is an authorization barrier, so streams established during setup send no messages until the response accepts the binding. Every reconnect rotates the nonce, clears the old answer, and renegotiates it. The runtime emits the existing v0 heartbeat every 500 ms after registration; a revoked key therefore severs the transport and reaches the core-owned partition/hosted-run hold path.

A runnable remote SDK advertises waddle.v0.hosted.runs. Accepted requests open through the ordinary reset/recording path; duplicate IDs return the original admission result. Requests and statuses never buffer across connections. Timeout or disconnect requests core HOLD and aborts the episode, and reconnect never replays motion.

RGB-D calibration resolves the selected pixel against the exact latest local depth frame and transmits only the bounded 3-D measurement. Image/depth arrays remain customer-side. Guided calibration orchestration belongs to applications and the hosted UI, not to an SDK-local web server.

Robot modules (waddle_sdk.robots)

A vendor module is facts plus a lazy driver factory over the vendor-neutral robots.base layer. Factory construction opens no bus and starts no thread; Site.open() owns the actual lifecycle. The built-in YAM, xArm 6/7, Alicia-M, Alicia-D, MuJoCo, ROS 2, and camera adapters import vendor SDKs lazily, so ordinary imports require no hardware packages. The three manifest-native physical families expose a single joint space with a normalized 0..1 gripper row; Applications own IK and planning above this boundary. For no-hardware work, waddle_sdk.robots.mock:arm is a manifest-native configurable simulated arm with planar FK and conservative body geometry. waddle_sdk.robots.mujoco:arm keeps the compatible private-world factory. waddle_sdk.robots.mujoco:backend is the shared-world form: it loads one site-relative MuJoCo XML model for every attached part and camera, advances that world once per SDK tick, renders aligned RGB-D, and evaluates TCP and conservative body geometry on a separate scratch state before dispatch.

The owner supplies hard safety. base.Arm checks declared width, finiteness, joint limits, per-step travel, optional workspace bounds, and SDK-owned static keep-out/body-collision rules over adapter-supplied conservative spheres. It rejects a complete target and never clamps it; malformed or missing configured geometry is itself a refusal. Posture chooses which verbs a driver can expose (monitor or supervised); it never decides who holds authority. Claims, leases, and handoff remain native internals.

Managed cameras implement capture() -> CameraFrame plus idempotent close(). Each RGB/RGB-D sample receives one paired monotonic/Unix stamp, RGB follows the recording/media path, and aligned depth remains in the SDK process.

Write your own vendor module

For the complete porting boundary, optional runtime facets, fallback behavior, and omission matrix, see Porting a hardware or simulator backend.

A driver is any object with the ten members of base.Driver (kind, estopped, read, write, hold, estop, re_enable, step, home, close) — a typing.Protocol, so yours is admitted on its members and never on its ancestry. kind is your driver's own word for itself, and this layer reads it in ONE direction: "sim" alone selects the harmless branch of the two questions it asks (does closing this drop all torque, is homing it a motion nobody is watching), and every other word is treated as metal. Drivers that can atomically command a position and a known trajectory velocity may additionally implement write_position_velocity(target, velocity_feedforward_rad_s) -> bool. Returning false means the driver intentionally issued its position-only fallback. This is an optional extension and is not added to base.Driver. Both protocols are also exported directly from waddle_sdk.robots for custom packages; no inheritance or YAM dependency is required.

The rest is a facts table, a driver and a factory. These are the exact lines tests/test_robots_base.py builds a whole toy vendor module out of and drives end to end through a real session — declaration, envelope, gate, pump, MCAP read-back — with nothing vendor-specific in base to help it:

import waddle_sdk
from waddle_sdk.robots import base

TOY_FACTS = {
    # The vendor's own numbers, with their provenance in the comment beside
    # them in a real module. A toy crane: two arm joints and a hand.
    "joints": ("boom", "stick", "grip"),
    "limits": ((-1.0, 1.0), (-1.5, 1.5), (0.0, 1.0)),
    "step_caps": (0.10, 0.10, 0.25),
    "max_effort_nm": 4.0,
    "rate_hz": 20.0,
    "home": (0.0, 0.0, 1.0),
}


def toy_driver() -> base.SimDriver:
    return base.SimDriver(
        TOY_FACTS["home"],
        lower=[lo for lo, _ in TOY_FACTS["limits"]],
        upper=[hi for _, hi in TOY_FACTS["limits"]],
        step_caps=TOY_FACTS["step_caps"],
        rate_hz=TOY_FACTS["rate_hz"],
    )


def toy_crane(*, posture: str = "supervised") -> base.Rig:
    """The whole of a second vendor module: declare the robot, say how to open
    it, hand back a rig."""
    space = waddle_sdk.descriptors.JointSpace(
        joints=[
            waddle_sdk.descriptors.Joint(
                name=name,
                min_position=lo,
                max_position=hi,
                max_effort=TOY_FACTS["max_effort_nm"],
            )
            for name, (lo, hi) in zip(TOY_FACTS["joints"], TOY_FACTS["limits"])
        ],
        rate_hz=TOY_FACTS["rate_hz"],
        chunking=waddle_sdk.descriptors.Chunking(
            horizon=1, replan="immediate", interp="hold"
        ),
    )

    def build_arms() -> dict[str, base.Arm]:  # the bus opens HERE
        return {
            "": base.Arm(
                part="",
                driver=toy_driver(),
                joint_names=TOY_FACTS["joints"],
                joint_limits=TOY_FACTS["limits"],
                step_caps=TOY_FACTS["step_caps"],
                rate_hz=TOY_FACTS["rate_hz"],
                home_values=TOY_FACTS["home"],
            )
        }

    return base.Rig(
        declaration=waddle_sdk.descriptors.Robot(
            name="toy-crane", robot_id="toy-crane-01", action_space=space
        ),
        build_arms=build_arms,
        rate_hz=TOY_FACTS["rate_hz"],
        posture=posture,
    )

That is the whole of a second vendor module, and it is a test rather than a docs snippet in the only sense that survives a year: the claim it makes — "the base layer carries all of the behaviour" — is true exactly while that test passes, and the block above is not a retelling of it but the same text, held to the test's own source by test_the_published_template_is_these_same_lines. (The two imports are yours; every line below them is that file's.)

toy_driver() returns the shipped twin because a test has no bus to open. Yours returns your own driver there — any object with the ten members above — and a module that ships both takes sim: bool and branches, as waddle_sdk.robots.yam does. Either way it is constructed inside build_arms and never at import, so the factory call opens no bus and starts no thread: a program may build a rig and then decide not to run it.

For more than one part, declare a waddle_sdk.Composite and return one Arm per part name (declaration order IS the concatenated action layout). Add fk= to each Arm if you have forward kinematics, and only then a workspace= box. To enforce manifest keep-outs or self-collision, also pass collision_frame="..." and a deterministic collision_spheres(q) callable that returns conservative named base.CollisionSphere values for every robot body the rule protects. The callable reports geometry only: SDK code owns intersection, margins, ignored adjacent-body pairs, whole-command refusal, and cross-part atomicity. Ship the source that can gate a fact next to the facts — waddle_sdk.robots.yam vendors the model its numbers come from (MIT data inside this Apache-2.0 wheel: Third-party content in the wheel) and tests/test_yam_facts.py compares every one of them against it, directionally (a declared limit may only be TIGHTER than the model's) — and where nothing can gate a number, say in the comment which pinned artifact it came from. An unsourced number is one nothing checks.

Installing

pip install waddle-sdk                    # control plane included
pip install 'waddle-sdk[orbbec]'          # + lazy Orbbec RGB-D adapter
pip install 'waddle-sdk[realsense]'       # + lazy RealSense RGB-D adapter
pip install 'waddle-sdk[usb]'             # + lazy OpenCV USB/UVC adapter
pip install 'waddle-sdk[cameras]'         # + Orbbec, RealSense, and USB
pip install 'waddle-sdk[xarm]'            # + UFactory xArm SDK
pip install 'waddle-sdk[alicia]'          # + Alicia-M SDK (Python 3.11+)
pip install 'waddle-sdk[alicia-d]'        # + Alicia-D SDK (Python 3.11+)
pip install 'waddle-sdk[robots]'          # + all three physical families
pip install 'waddle-sdk[mujoco]'          # + MuJoCo 3.5+ simulation/scene compiler
pip install 'waddle-sdk[media]'           # + the LiveKit media plane
pip install 'waddle-sdk[cameras,media]'   # camera adapters + LiveKit media plane

Two distributions from one source tree (the psycopg / psycopg-binary shape): waddle-sdk bundles the core built with the grpc control transport, and the media extra adds the exact-pinned waddle-sdk-media wheel, which is the SAME shim built with livekit too. LiveKit's libwebrtc dependency chain is ~690 MB of build, and an install whose job is to supervise a policy should not pay for a LiveKit media plane it will never open. Either way you import waddle_sdk: waddle_sdk._native picks the richer core whenever its binding API is compatible. A package-version mismatch warns but still uses that compatible media core; only a binding-API mismatch falls back to the bundled core. WADDLE_NO_MEDIA=1 remains an explicit opt-out.

Third-party content in the wheel

This repo is Apache-2.0 and the wheel carries one deliberate exception: waddle_sdk/robots/yam_data/ is a vendored snapshot of I2RT's YAM robot description, shipped under its own MIT licence (yam_data/LICENSE, verbatim from the source repo) and pinned to the upstream commit its README names. It is data rather than code, text only — the STL meshes are not shipped — and it earns its 16 KB twice: waddle_sdk.robots.yam hands it to a single-arm declaration as kinematics_urdf, and tests/test_yam_facts.py compares every constant in that module against the vendor's own numbers in it.

Development

cd sdk
uv sync --dev --extra mujoco && uv run --no-sync pytest              # full build + test
uv run --no-sync maturin develop --uv && uv run --no-sync pytest   # iterate on Rust
cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
cargo fmt --manifest-path rust/Cargo.toml --check

The full test suite uses the optional MuJoCo scene compiler. On headless Linux, set MUJOCO_GL=egl before running rendering tests. The base SDK remains usable without MuJoCo.

The shim is its own cargo workspace (rust/) with path-deps into ../waddle-core/crates/*; pyo3's extension-module feature lives only in [tool.maturin].features so plain cargo check/clippy keep working.

Connected builds

The transports are cargo features on the shim. grpc (the control plane: create_session(transport_url=, transport_token=)) is in the DEFAULT build, so uv sync --dev/maturin develop already carry it; livekit (the media plane: media_url=, media_token= — the plane mints both tokens, this SDK never does) is not, since it belongs to the waddle-sdk-media companion:

uv run maturin develop --uv --features grpc,livekit    # the companion's flavour
cargo clippy --manifest-path rust/Cargo.toml --features grpc,livekit --all-targets -- -D warnings

Clippy must be clean featureless, --features grpc, and --features grpc,livekit. A build that lacks a feature refuses the matching kwarg rather than running offline in silence. waddle_sdk.FEATURES is the probe, not waddle_sdk._core.FEATURES: _native selects which core this process runs on and re-exports that core's features, so on a [media] install waddle_sdk._core.FEATURES still reports the bundled core's grpc-only set while the process is running the media core. It is the only feature detection the Python layer does, and the package reads it only at native construction to refuse a transport= or media= this build cannot honour.

The companion wheel is sdk/media/pyproject.toml: same manifest-path = "../rust/Cargo.toml" (hence the same version by construction), module-name = "waddle_media._core", features grpc,livekit. sdk/pyproject.toml's [tool.uv.sources] points the media extra at it so uv can resolve the lock; nothing builds it unless the extra is actually installed. The extra's exact pin (waddle-sdk-media==X) is the ONE version here that maturin cannot derive from the manifest, so a version bump must edit it too — tests/test_features.py holds it to waddle_sdk.__version__ (and the two projects to one manifest) rather than to memory.

Opt-in real camera publication acceptance

tests/test_livekit_public.py exercises Site.open(media=LiveKit(url, token)) with a synthetic camera and arm, a real LiveKit service, and an independent RTC viewer. Install the matching [media] companion plus the test-only livekit==1.1.18 and websockets==17.0.1 clients. The caller supplies three explicit environment values: WADDLE_TEST_LIVEKIT_URL, WADDLE_TEST_LIVEKIT_PUBLISHER_TOKEN, and WADDLE_TEST_LIVEKIT_VIEWER_TOKEN. The tokens must name the same fresh sdk-media-test-<unique-id> room with different identities: publisher-only camera permission and viewer-only subscribe permission. These externally supplied grants leave room creation/deletion with the caller. Alternatively, supply WADDLE_TEST_LIVEKIT_URL, WADDLE_TEST_LIVEKIT_API_KEY and WADDLE_TEST_LIVEKIT_API_SECRET; the selected media fixture creates an isolated room, issues scoped grants and deletes that room afterward. Collection and non-media selections issue no service requests. Without --live and either credential form, it skips before dialing an endpoint. This synthetic test opens no physical hardware. Never put signing keys or grants in logs.

python -m pytest --live -q tests/test_livekit_public.py

The test receives RGB and colorized <camera>/depth frames, checks the published source resolution and separately confirms local metric depth. Received video may adapt resolution to network conditions; reports distinguish those dimensions. A loopback-only signaling proxy drops this test publisher's sockets; the native transport reconnects while the same public SiteSession and entered Run remain owned. A new viewer then rediscovers both tracks, followed by normal public SDK shutdown. Proxy handshake logging is disabled because headers can contain scoped tokens. This proves media transport and session continuity, not browser rendering or physical-camera quality.

With the same credential options and a configured physical site, run python -m pytest --live --live-config=/absolute/bench.json tests/live/test_03_media.py. This tests each selected real camera through the same SDK publication/reconnect path, using a mock motion context without opening its physical arm. Camera ownership retains the original site ID. See the live test guide for discovery and evidence requirements.

LiveKit room and publisher identity come from the caller's grant and remain stable for the opened session. Native LiveKit manages reconnection and server-issued token refresh. The public SDK currently has no API to replace an opened session's media URL, room, identity or caller-provided grant. Do not reopen a hardware site merely to hand off an application's unrelated connection; retain the SDK owner and its media publication.

Hollow-frontend checklist

All claim, lease, handoff, gate, timeline, and timestamp decisions live in waddle-core exactly once.

• SiteSession owns construction and cleanup, not authority. It never stores claim IDs, lease IDs, gate modes, or connection-derived permissions.
• Run.step marshals one native gate result into the owner envelope. It may index the part named by core and reject the whole owner command; it never decides which actor may command it.
• SiteSession.hold and estop call native priority paths. They do not invoke driver callbacks on the caller thread or fabricate timeline events.
• SessionClock remains the only production source of paired timestamps. Camera samples and runtime events copy native stamps.
• gRPC and LiveKit URLs/tokens are constructor data. Python consults only _native.FEATURES, a build fact, and never infers connection state.
• Calibration deprojects local aligned depth and submits one bounded point. It grants no motion and carries no image through the control plane.
• Driver modules may enforce the owner's physical envelope and lifecycle. They contain no claim, lease, hosted-task, workspace, or application orchestration logic.

Private _testing loopback hooks remain only for core conformance tests. They are not exported by the package and are not an alternate application API.

Optional gripper force actuation

An opened part can advertise actuation.gripper_force and observation.gripper_force. Its describe().command_limits[part].gripper_force_n declares the supported Newton range, and observe_parts() includes a sourced GripperForce reading. Custom hardware implements the structural robots.GripperForceDriver extension. Kinematic simulators cannot supply force.

A named JointPositionCommand(positions, gripper_force_n=10) activates the supported jaw force servo after ordinary native and owner-envelope admission. The position reference bounds approach travel; the driver maintains force when contact is acquired. An omitted modifier preserves force mode during arm writes; gripper_force_n=0 returns to position control. Hold, e-stop, close and a selected supervision action clear the mode. Callers own measured acquisition decisions; the SDK introduces no grasp-completion or convergence policy.

YAM LINEAR_4310 feedback uses CAN-reported gripper_eff in Nm, inferred from motor current. Its nominal conversion is closing_torque * 6.57 / 0.096 N. The native MuJoCo YAM adapter uses its driven slide actuator force, conjugate to total jaw separation. Both readings explicitly report estimated=True and a conversion source. These are actuator-effort estimates: friction, transmission loss and transient inertia require separate physical calibration before claiming accurate object contact force. No load-cell calibration or physical force acceptance is established by software or native simulation tests.