lekit-robot

0.5.0 observation API: body observations are always included. Select camera streams explicitly with cameras={"head_view": ["color", "depth"]}. This replaces the 0.4.1 observation options; see CHANGELOG.md for migration details.

Python SDK for a running Lekit Robot App, requiring Python 3.10+. Each application owns its SDK, subscriptions and caches. There is no shared SDK server. The client does not install Robot App, Web, LeRobot, Torch, OpenCV or hardware drivers.

Install

Install or upgrade the client from PyPI (Python 3.10+):

python -m pip install --upgrade lekit-robot==0.5.0
# Include RGB/depth image decoding:
python -m pip install --upgrade 'lekit-robot[images]==0.5.0'

Protocol and transport implementations are included inside this SDK distribution. No separate lekit-robot-protocol or lekit-robot-transport PyPI packages are required.

Omit [images] for body state and metadata. NumPy is a base dependency; JPEG/PNG and LZ4/Zstandard codecs are optional. Public camera observations always return decoded arrays, so install [images] when selecting camera streams.

Build the single SDK distribution from the repository root:

uv build --package lekit-robot --out-dir /tmp/lekit-client-wheels
python -m pip install /tmp/lekit-client-wheels/lekit_robot-0.5.0-py3-none-any.whl

For editable development, use the repository workspace, which supplies the canonical shared sources used by Robot App as well:

uv sync --all-packages

Robot App must already be running and advertise name discovery. Connect by its configured robot name on the same LAN; no URL is needed. Zenoh multicast discovery must be reachable. Routed VPNs or networks that block multicast are not discovered automatically. A loopback-only App remains accessible only on its own host. Updating SDK packages does not deploy or restart a robot.

What's new in 0.5.0

• observation() defaults to body state. A single camera-name mapping selects color, depth or both, and selected images always decode to arrays.
• Quality checks are strict and timeouts raise; obsolete observation switches and invalid-cache timeout fallback are removed from the public method.
• after=previous and freshness="all" retain explicit new-sample semantics. connect("robot name") and advanced DeviceObservations remain available.
• Control receipt correlation and timeout handling are hardened without replaying uncertain actions or treating local submission as arrival.

This release targets the current Robot App architecture and control protocol 3. Applications using the old server-aligned observation mode must migrate for device-source operation. Full release notes are included as CHANGELOG.md in the source distribution.

Public application API

from lekit_robot import connect

with connect("NERO Dual") as robot:
    meta = robot.meta_info
    previous = robot.observation()  # Body only; no image demand.
    current = robot.observation(after=previous, freshness="all", timeout=.5)
    print(meta.manifest.robot_name, current.robot.state)
    print(current.quality.ages_s)
    print(current.control.feedback)

basic_api.py demonstrates the complete lifecycle. It runs read-only unless an action file is supplied:

python packages/lekit_robot/examples/basic_api.py --name "NERO Dual"

The demo's observations read only the body and need no image extra. Passing --action-file target.json explicitly enables its control loop; --steps sets the number of iterations. The loop targets 60 Hz but observation, computation and scheduling share that budget. It does not treat local submission or a stop ID as physical completion.

For a finite, observation-only example, use read_observation.py:

python packages/lekit_robot/examples/read_observation.py --name "NERO Dual" --mode body --count 5
python packages/lekit_robot/examples/read_observation.py --name "NERO Dual" --mode rgbd --camera head_view --count 5

Omit --camera to select all advertised cameras in image modes. This example requires every selected source to advance after the first observation, reports quality and per-source ages, and closes subscriptions automatically. Use --full-state to print the complete body state dictionary.

NERO joint torque

An updated NERO Robot App includes joint torque in the same body samples as joint positions. The existing client reads it from observation.robot.state:

with connect("NERO Dual") as robot:
    previous = None
    for _ in range(60):
        previous = robot.observation(after=previous, freshness="all", timeout=.5)
        state = previous.robot.state or {}
        key = "left.joint_1"  # left/right, joint_1 through joint_7
        torque = state.get(key + ".torque")
        if state.get(key + ".torque_feedback_fresh"):
            print(state.get(key), torque, state["joint_torque.unit"])

joint_torque.estimated=True identifies the vendor SDK's current-derived estimate in N·m, not a torque sensor or an external-contact torque measurement. Each joint includes .torque_feedback_timestamp_s (device-host Unix receive time), .torque_feedback_age_s (age when sampled), and .torque_feedback_fresh. Missing, nonfinite, expired or disconnected feedback has no .torque value; zero is a valid measurement. Torque freshness is independent of position and the overall observation quality. Use the per-joint flag, and do not subtract this device timestamp from a different host's clock.

Torque is read directly from the latest motor feedback on every body sampling cycle, bypassing the slower diagnostic snapshot. It uses the existing body stream, with no extra HTTP request or subscription. A 60 Hz body stream does not guarantee 60 distinct torque readings if the motor feedback is slower; compare the per-joint timestamps to detect updates. Deploying the server change is required; installing the client alone cannot add absent server fields.

Production discovery and process boundaries

connect("NERO Dual") queries the LAN discovery responders for that robot name (trimmed and case-insensitive; other characters must match). It rejects duplicate names and conflicting robot/instance identities instead of choosing the first response. The announcement supplies robot_id, process instance, generation and API addresses; the caller supplies no URL. Discovery is part of each App process, not a central registry or an additional process.

The client reads GET /api/robots/<robot-id>/manifest at a discovered address, with the ID encoded as one URL path segment. The unified manifest describes identity, capabilities, sources and interfaces. It must match the discovery announcement before the SDK registers a control session. An explicit mode="device-sources" selects the production source architecture:

1. Project the source directory from manifest.sources, using the same manifest's embedded body identity; verify its manager, body instance and generation. This does not require separate directory and v1 manifest reads.
2. Establish the original App control backend. auto / zenoh read /api/robots/<robot-id>/realtime and verify the control identity; an explicit endpoint= changes that control endpoint, not the individual source addresses.
3. On the first observation(), connect directly to the selected body and camera sources. Validate each descriptor and clock bound, then combine in this process.

Robot App's public robot APIs use /api/robots/<robot-id>/<api-name>, including configuration, diagnostics, observation and control. Schema versions remain in payloads. /api/robots/<robot-id>/legacy-manifest exposes the old v1 body-only contract; it is distinct from the unified /manifest document.

Only HTTP 404 during initial read-only discovery permits trying the older /api/device-manifest and /api/manifest/v2 aliases. If no unified manifest is available, older services can use /api/transport, /api/devices and /api/manifest as the compatibility description path. Invalid documents, identity conflicts and non-404 responses do not select an older alias. Once the robot-scoped API is verified, later reads and actions stay bound to that robot; a missing scoped endpoint never downgrades to an unscoped default robot.

The body source runs inside Robot App beside local feedback sampling and control. Each camera child owns its acquisition, RGB-D pairing, encoded cache, source publisher and finite subscription leases. App manages their lifecycle and the read-only directory; it does not default to receiving and rebroadcasting all images. Source listening ports come from the directory and are not all port 7447. Zenoh is the current replaceable network adapter; no router process is required. Web uses a fixed App HTTP/WebSocket upstream. Its browser first reads the Web-owned /api/robots list, then the selected robot's scoped manifest; this single-upstream list is not automatic Fleet discovery. Previews connect directly to the camera MJPG ports declared in the manifest. Those camera-owned endpoints retain their own URLs. Web does not import this SDK or combine observations.

For compatibility or debugging, connect(url="http://ROBOT_IP:8000") explicitly selects an API endpoint; adding name="NERO Dual" checks its name. A Web backend that proxies the API is also valid, while a static UI server alone is not. Neither omitted arguments nor failed name discovery fall back to localhost. endpoint= overrides the control middleware endpoint after resolving the App; it is not a robot-discovery address. HTTP-only or older Apps without the name_discovery capability still require this explicit URL path.

transport="http" explicitly selects body-only compatibility: use observation(). It does not reconstruct mixed body/camera observations through App. auto falls back to the original HTTP control backend only when realtime discovery returns 404; transport failures do not silently change routes or replay motion. An App declaring integrated-body, or a legacy App without the new mode, keeps its own advertised compatibility behavior. New static diagrams and SDK installs are not evidence that a particular running App has been upgraded.

Source subscription and observation contract

The public method has one explicit camera selection and a small set of quality and new-sample controls:

def observation(
    *,
    cameras=None,      # Mapping of camera name to ["color"], ["depth"] or both.
    timeout=3.0,
    max_age=None,      # None means .25 seconds; scalar or per-source mapping allowed.
    max_skew=.05,
    after=None,
    freshness="any",
): ...

with connect("NERO Dual") as robot:
    body = robot.observation()
    rgb = robot.observation(cameras={"head_view": ["color"]})
    rgbd = robot.observation(cameras={"head_view": ["color", "depth"]})
    mixed = robot.observation(cameras={
        "head_view": ["color", "depth"],
        "left_view": ["color"],
    })
    print(rgbd.camera("head_view").color.shape)

The body is always included. Omitted cameras, None and {} mean body only. Each camera name must be unique in the manifest and each value must explicitly select supported streams; unknown names, ambiguous names and unsupported streams fail. No camera or stream is added implicitly. Images are decoded NumPy arrays. To select all cameras, build a mapping from robot.meta_info.manifest.cameras; requesting RGB-D from a camera without depth fails rather than dropping depth.

This replaces the old images, depth, robot, mode, at, strict and on_timeout keywords and the camera-name sequence. Those arguments are no longer accepted by RemoteRobot.observation(). For metadata-only, encoded-image, camera-only or other advanced source workflows, use the separate connect_devices() / DeviceObservations interface described below.

The following describes connect() against device-sources:

• One active source/stream selection is retained per client. Repeated calls reuse its subscriptions and bounded latest caches. Changing source or stream selection closes the old set before opening the new one. Image subscriptions remain active between calls until selection changes or the client closes; there is no idle automatic unsubscribe. Lost clients are bounded by source lease expiry.
• Without after, a call may return the same fresh cached samples. after=previous must belong to this client’s current subscription and source/stream selection. freshness="any" requires at least one selected source to advance; freshness="all" requires every source to advance and is limited by the slowest source. This is not an instruction to generate samples at the caller's rate.
• Sources have independent identity, instance, generation, sequence and clock. RGB-D blocks belong to the same camera frameset. Source replacement requires reconnecting; old cursors, sessions and actions never silently rebind.
• Only latest sampling is supported by this public method. Latest software combination is not hardware synchronization or a lossless recording API. There is no cross-device atomic configuration revision.
• Default maximum ages are 250 ms for body and cameras; maximum skew is 50 ms. Override all sources with max_age=.2, or use max_age={"robot": .1, "camera": .2, "head_view": .15} or source IDs, and max_skew=.03. Quality is checked again at delivery after decoding and management refresh. timeout also bounds subscription setup and serialized waits; it is not a hard realtime completion guarantee.
• quality.ages_s contains capture-age upper bounds by source ID, using separately measured clock-offset intervals. receipt_ages_s measures local time since receipt; clock_uncertainty_s reports clock interval width. Cross-domain skew uses conservative transformed intervals. Host monotonic clocks are never subtracted directly. Software clock bounds assume 1 ms plus 100 ppm drift allowance and expire after 10 seconds; they are not hardware clock sync.
• Quality validation is always strict and timeout always raises ObservationUnavailableError; the public method never returns an invalid old observation as a timeout fallback. Identity, lease and clock failures also cannot fall back to old data.
• Returned source_cursors and source_calibrations preserve per-source evidence. The explicit connect_devices() client also exposes its selected descriptors through devices. The body carries its sampled App config revision. Camera-only results from the explicit device API use zero rather than inventing a global revision. Metadata refresh happens during observation delivery when needed; property reads remain local.

The App's bounded HTTP/LOBS compatibility snapshots remain separate. Some old services support server-side aligned/historical selection and idle stream release; those semantics do not apply to device-source subscriptions. See the device API, source contracts and middleware contract.

Explicit read-only device API

connect_devices() accepts an App base URL or a DeviceDirectory snapshot. For name-based discovery without a control session, obtain the unified manifest with the SDK discovery helper and project its directory as shown below. connect_devices() opens a fixed set of source subscriptions immediately and creates no control session. Use the actual registered IDs; production body IDs are body:<robot_id>.

from lekit_robot import connect_devices
from lekit_robot.discovery import discover_manifest

found = discover_manifest("NERO Dual")
directory = found.manifest.source_directory()
body_ids = tuple(entry.device.identity.device_id
                 for entry in directory.devices if entry.device.kind == "body")
with connect_devices(directory, source_ids=body_ids, streams=()) as sources:
    previous = sources.observation(images=False)
    current = sources.observation(after=previous, freshness="all", images=False, timeout=.5)
    print(current.robot.state, current.source_cursors)

streams=() creates no image demand. For this fixed-selection API, observation(images=False) skips decoding but does not change already selected image streams or release their leases. Choose streams when connecting and close when finished. It has latest/after any/all quality checks but no timeout-reuse option, actions or automatic re-registration after source replacement.

Reading the directory does not establish subscriptions or wake cameras. Established subscriptions do not need directory HTTP on every frame. However, production App owns body and camera lifetimes: App exit closes the body source and stops camera children. The separate-manager simulation's sources surviving manager exit is not a claim that production cameras survive App exit.

Read-only diagnostics and examples

To show this SDK instance in /monitor → “观测链路”, explicitly enable robot.start_observation_diagnostics(label="policy") after connect("NERO Dual"). It reports bounded counters and timing summaries every two seconds over the existing management session; enabling reporting does not start observations or wake cameras. Continue calling observation() in your application. robot.observation_diagnostics() reads a local snapshot and robot.stop_observation_diagnostics() removes the report without stopping the observation subscription. Closing the robot also cleans up reporting. Requires the App's observation_diagnostics.version=1 source-middleware capability.

python packages/lekit_robot/examples/diagnostics_observation.py --name "NERO Dual" --seconds 30 --mode rgbd

The flow view separates source capture/send from SDK receive/decode/return and all-source advancement. Reports are client measurements, not server certification; stale or absent reports are explicitly labeled. See observation diagnostics for clocks, bounded windows, failure categories and cleanup semantics.

lekit-robot --help
python -m lekit_robot --help
lekit-robot --name "NERO Dual" --observation --mode robot --count 300
lekit-robot --name "NERO Dual" --observation --mode rgb --camera head_view
lekit-robot --name "NERO Dual" --observation --mode rgbd --verbose
python packages/lekit_robot/examples/benchmark_observation.py --name "NERO Dual" --mode robot --count 300

The benchmark example waits for every selected source to advance before counting a new observation (after=previous, freshness="all"). It reports return intervals, unique updates per source and skipped sequence numbers separately. It does not add a second 60 Hz sleep after waiting for the devices. Use --poll-latest --hz 60 only to measure periodic cache reads; repeated reads do not demonstrate 60 Hz fresh delivery.

The CLI runs on the client computer and never sends motion. --hz sets the application's target call rate, not the device capture rate. Source subscriptions remain active between calls; each call does not create a new request/reply stream. The default is 60 Hz and all configured RGB-D cameras. Choose --mode robot for no image demand, or install [images] for rgb/rgbd. These modes construct an explicit camera/stream mapping after connection; rgbd requires both streams on each selected camera. --camera accepts a name or ID and is repeatable; IDs are resolved to unique camera names. Ctrl+C / SIGTERM closes the client; --count bounds calls. The positional form lekit-robot "NERO Dual" --observation is also supported. Provide a name or an explicit --url; there is no default local robot.

The dashboard reports return Hz, call percentiles, per-source updates and repeats, failures, CPU and memory in a bounded three-second window. CPU 100% is one core. Returned count and update Hz are not capture frequency, and repeats are not proof of dropped frames. Diagnostic work adds overhead.

In device-source mode, displayed sample age is the SDK's capture-age upper bound. Legacy selection-time differences are a separate metric. Optional --clocks-synchronized enables client UTC minus source capture UTC estimates; it is a user assertion, not automatic clock synchronization. Negative estimates are labeled clock error and excluded from percentiles. Hardware CAN message age is separate from the body snapshot age.

The CLI has no encoded or aligned-read mode. Public observation timeouts are failures, not successful reused returns. Its JSON labels source-mode age upper bounds separately from compatibility selection-time age. Neither tool certifies stable 60 Hz or real robot stopping.

For a hardware-free transport rehearsal, the repository provides:

.venv/bin/python scripts/exercise-device-services.py --seconds 60 --cameras 3

Its separate simulated processes send pre-encoded frames. It does not validate physical capture, per-frame encoding, hardware motion or another machine's network.

Control and safety

send_action() and stop() require control protocol 3. Both return an action ID following local enqueue, not device acceptance, dispatch, arrival or confirmed standstill. The application owns policy, pacing and recording.

• Connecting registers a session without occupying the robot. The first valid action claims whole-robot control; competing writers receive CONTROL_BUSY.
• Targets are absolute. Joint/gripper field order and units come from metadata. Pose supports {position, orientation, frame} with a unit xyzw quaternion, or {position, rpy, frame}. Groups come from the robot description rather than assuming every robot has left/right arms.
• Each scope keeps the latest pending target. There is no hidden periodic resend, transport retry, atomic multi-arm transaction or replay on reconnect. Action and stop carry ordered steps and session fences so delayed old targets cannot revive.
• App retains ownership, freshness, mode, fault, emergency-stop, communication, limit and generation checks, including checks before actual dispatch. IK and final safety decisions remain on the device, not in Web or the SDK.
• Accepted work survives client close. close() discards unsent work and does not stop accepted motion. Explicit stop() cancels targets and releases ownership only after confirmed hold; failures keep the safety barrier locked. Observe matching feedback before closing when delivery or hold confirmation matters.
• Normal idle release requires 2 seconds without an accepted action, all latest targets reached, and no pending dispatch or stop. The hardware retains its position target. A slow one-shot target is not cancelled merely after 2 seconds.
• Default execution protection is 30 seconds per unresolved target and 10 seconds without measured progress. This software policy is not a hardware emergency stop.
• obs.control carries local status, asynchronous feedback and local monotonic feedback-receipt age. It is copied at return, not aligned to sensor capture. Check per-part errors and rejection as well as the top-level device fault; an empty error does not prove completion.

See CONTROL_PROTOCOL.md and the complete basic_api.py demo for control confirmation. get_observation, iterators, explicit acquire/release and old target setters are not public compatibility aliases. Use observation, send_action, stop and meta_info. The wire namespace alone does not guarantee compatibility; upgrade App and SDK together and reconnect after instance or generation changes.

Distribution verification

Build/install the single SDK wheel, outside the source workspace with no PYTHONPATH. Verify imports, both CLI help entry points, the read-only demo, connect() and connect_devices() against an isolated Mock App. Use a separate image-extra environment to verify decoding. Package installation and Mock checks do not imply deployment, real camera throughput or physical safety acceptance. Publish only after separately verifying the intended registry/version; this repository's source version is not automatically a released artifact.

Release builds package the canonical shared sources under lekit_robot._vendor and relocate internal imports at build time. They neither install nor alias the top-level shared modules used by Robot App. The sdist includes the shared source and licenses and can rebuild without this repository. Use lekit_robot public types in SDK applications; serialized protocol compatibility is retained, while private bundled classes are not the same Python class objects as separately loaded App classes. Editable workspace development continues to share App types.

License: Apache-2.0.