lekit-robot

Small Python SDK for a running Lekit Robot Node. Discover a robot by name, read joint/pose feedback and RGB/depth images, and send absolute motion targets. The client does not install LeRobot, Torch, OpenCV, robot drivers, or camera drivers. Python 3.10+ is required.

Application API and one complete demo

The complete application surface is connect() plus four client methods and one property:

basic_api.py demonstrates this entire lifecycle in one file. Its loop belongs to the application. Run read-only from the repository:

PYTHONPATH=packages/lekit_robot/src .venv/bin/python \
  packages/lekit_robot/examples/basic_api.py --name "NERO Dual"

Supplying --action-file target.json enables the control section, using the explicit target for 60 externally paced iterations by default (--steps N). The Node must advertise control_protocol_version: 2; otherwise local action submission reports UNSUPPORTED_PROTOCOL and no motion is submitted. This demo does not treat local submission/stop as proof of physical arrival or standstill.

Old public methods/properties have been removed from RemoteRobot. This is a breaking change in 0.1.9. Use meta_info.manifest for identity, name, generation and target fields; use meta_info.calibration for calibration. with connect(...) as robot: is syntax for automatic close().

0.1.10 update

Heartbeat failures retry within the current session lifetime. Expired sessions recover automatically on the same Robot Node instance and robot generation, so a long-lived Robot object can accept new actions after communication recovers. Old actions are discarded; interrupted trajectories are never resumed automatically. Public method signatures and control protocol version 2 are unchanged.

Install

Install or upgrade to this release:

python -m pip install --upgrade lekit-robot==0.1.10

For development from this repository:

python -m pip install --upgrade --editable ./packages/lekit_robot

Reconnect running client programs after updating. Version 0.1.9 removes the old public entry points and explicit control acquisition/release methods. Update both the client and Robot Node for the implicit-control contract described below.

Robot Node must already be running on the robot computer. Clients need network access to its Zenoh peer; the browser's HTTP port 8080 is not the SDK endpoint. LAN multicast discovery is the default. If discovery is blocked, configure both Node (lekit-robot --zenoh-router tcp/ROUTER_IP:7447) and client (connect(name="NERO Dual", router="tcp/ROUTER_IP:7447")) to use an existing Zenoh router. This package installs the client CLI, not the Robot Node server. In a full lekit installation, lekit-robot-node starts the server; the former lekit-robot --host ... --port ... invocation remains compatible.

Continuous observation test

lekit-robot nero-dual --observation
# Exact advertised name; 30 Hz RGB-only observations over an explicit Zenoh endpoint:
lekit-robot "NERO Dual" --observation --router tcp/192.168.3.241:7447 --hz 30 --mode rgb
# State only, or restrict the selected cameras:
lekit-robot "NERO Dual" --observation --mode robot
lekit-robot "NERO Dual" --observation --camera head_view --camera left_view
# Compare with the previous 50 ms lookback selection:
lekit-robot "NERO Dual" --observation --observation-mode aligned

Run this on the policy/client computer to include its network path. It calls the real observation() API using persistent subscriptions, without acquiring control or sending actions. The Rich panel continuously reports achieved return frequency, call P50/P95/P99, startup, errors, repeated samples, period overruns, estimated source ages at return (latest and P95), source ages at T, skew, decode/cache metrics, CPU and memory. CPU 100% means one core; RSS is available on Linux. Measurement includes diagnostic overhead.

Default: 60 Hz, latest complete samples with timestamp validation, RGB and enabled depth from all cameras, runs until Ctrl+C. --observation-mode latest|aligned selects sample timing independently of --mode (image format/content). Use --hz, --mode rgbd|rgb|encoded|robot, --camera NAME (repeatable), --timeout .2 (selection wait seconds; decoding can extend total call time), --connect-timeout 3, or optional --count 300 for a bounded run. The panel refreshes twice per second; rates/percentiles cover a bounded three-second window. First successful call is shown separately from steady-state latency; counts are cumulative. Ctrl+C/SIGTERM closes the client and retains the final panel. Timeout reuse is counted separately and included in return frequency/counts; the panel marks reused observations as invalid instead of showing a fresh success.

The compact dashboard highlights return Hz, call P95/P99, and timeout-reuse percentage. With confirmed clock synchronization it also shows the maximum return age of the current observation (not a historical maximum). The source table shows age at T, observed sample-update Hz, and repeat percentage over the recent three-second window. The first sample has no predecessor and is excluded from update/repeat counts. Update Hz measures new samples seen by this test, not the sensor capture frequency; repeated samples do not imply dropped frames. Timeout reuse is a separate percentage of returned observations. Over-budget percentage covers non-startup returned calls exceeding the requested period; failures are counted separately. These percentages display — without samples.

Use --verbose to show decoding/decompression timings, subscription reception, cache hits, CPU/memory, startup latency, and cumulative overrun/repeat counters. The default dashboard hides uncalibrated age columns and shows one clock notice.

The positional name first matches the advertised name (case insensitive). If not found, a hyphenated CLI name such as nero-dual retries as nero dual. Ambiguous names are rejected. --obervation is also accepted as a compatibility spelling. python -m lekit_robot ... is equivalent to the console command.

The observation freshness table distinguishes two ages:

• Return age ≈: client UTC at observation() return minus the selected sample's Node capture UTC. It includes transport/decode waiting, plus the 50 ms lookback when using aligned. The adjacent P95 covers successful observations in the last three seconds. Reusing a sample increases its return age.
• Alignment difference (对齐差值, formerly T 时年龄): alignment target T minus sample time in the Node clock domain. In latest, T is the newest known selected-source timeline; in aligned, it includes the lookback. Age at T is not capture-to-return latency. For example, T = 100 ms and sample time = 90 ms gives a 10 ms difference. T is the selection reference, not the current client time; a zero difference does not imply zero latency. The dashboard includes a visible metric legend explaining this distinction, percentiles, updates, repeats, and timeout reuse.

By default, return age and its P95 are hidden, one 未校准 notice is shown, and UTC differences are not included in statistics. Age at T, cross-source skew, call duration, and monotonic timeout-reuse duration remain available without cross-host clock synchronization.

After confirming that the client and Node clocks are synchronized by the system (for example, chrony/NTP), explicitly enable return-age estimates:

lekit-robot "NERO Dual" --observation --clocks-synchronized

This flag is a user assertion, not automatic clock calibration or verification. Return age remains an estimate; residual clock offset affects even positive values. Missing/timezone-naive capture times display —; negative ages display 时钟偏差 and are excluded from percentiles. Cross-host monotonic clocks are never subtracted. Camera ages describe frame groups; robot ages describe Node snapshots, not each underlying CAN message. On observation failures the table is labeled as the last successful observation. Node loss terminates the test; rerun after reconnecting.

The observation test command is available in lekit-robot 0.1.6 and later. Return-age freshness metrics require 0.1.7 or later. The default latest selection and --observation-mode require 0.1.8 or later. Timeout fallback with reuse markers, the explicit --clocks-synchronized gate, and the compact dashboard with --verbose require 0.1.9 or later.

Three primary interfaces

Version 0.1.9 provides the simplified observation(), send_action(), and meta_info interfaces in the existing package. send_action() returns a local ActionSubmission without waiting for Node. It requires a connected session and a Node advertising control_protocol_version: 2. Update/restart Robot Node with the matching implicit-control implementation before using actions; update both ends together. Older running Nodes return UNSUPPORTED_PROTOCOL. No synchronous compatibility alias is provided. Old public entry points and the target_ttl connection option have been removed; new targets have no update deadline.

from lekit_robot import connect

with connect(name="NERO Dual") as robot:
    meta = robot.meta_info                   # cached; no network query here
    obs = robot.observation()                # robot + all configured camera streams
    print(obs.robot.state, obs.quality)
    print(meta.manifest.joint_target.fields) # ordered names, units, bounds
    print(meta.calibration)                  # None if no calibration is available

    # Other selections, using the same persistent subscriptions:
    state = robot.observation(cameras=(), images=False)
    images = robot.observation(cameras=["head_view"], robot=False, depth=False)

Observation contract

• Default: decoded RGB and enabled depth, robot state, all advertised cameras, strict=True. images="encoded" retains compressed frames; images=False returns sample metadata without decoding; depth=False omits depth payloads.
• mode="latest" (default) selects the newest complete sample from each requested source, without a fixed lookback. Selection happens under one buffer lock and validates timestamp skew and age before returning. Camera metadata, RGB and requested depth must belong to the same frame group; if a newer group's payload is missing, the newest older complete group can be used, subject to those checks. T is the newest known timestamp among the selected sources, including robot message preparation timestamps. Excessive skew waits for an update; a selection timeout uses the marked fallback described below, without disabling alignment checks.
• mode="aligned" retains the previous 50 ms lookback from the latest received robot timeline (or selected cameras when robot state is excluded). Each source contributes its newest capture at or before T. Strict aligned mode waits for the selected frame's payload instead of falling back to an older complete frame. Robot timestamps describe the SDK snapshot read, not simultaneous acquisition of every CAN feedback field. Alignment is not hardware synchronization.
• at=T uses seconds in the connected Node monotonic clock, not client time.monotonic() or Unix time. An explicit at always uses causal aligned selection at exactly T, even with the default mode; it adds no lookback. For example, use a prior observation's aligned_at_monotonic_ns / 1e9. Only already buffered history can be selected.
• Defaults: age at T <= 250 ms, source timestamp skew <= 50 ms. Override with max_age={"robot": .1, "camera": .2, "head_view": .15} and max_skew=.03. Local time since the source last arrived also detects a stalled subscription.
• Missing/stale sources, mismatched image payloads and excessive skew wait for updates within timeout. By default on_timeout="reuse" returns the last valid observation for the same source/format/mode/quality-limit configuration on selection timeout. It retains original sampling timestamps and T, sets obs.quality.reused=True and valid=False, and exposes the elapsed time since the original valid return as reuse_age_s. reasons explains the failed attempt. Age at T still refers to the original T; capture-to-return age continues to grow. Repeated fallback never resets this clock or promotes the fallback to valid. A fresh valid observation automatically resumes normal delivery.
• Use on_timeout="raise" to preserve fail-on-timeout behavior. First-use timeouts, unmatched fallback configurations, and explicit at=T queries still raise ObservationUnavailableError. Decode errors are not treated as selection timeouts and still raise immediately in strict mode. Its .quality.reasons explains the failure; .quality is None if no target could be established. strict=False permits partial/invalid results; always inspect obs.quality.valid in that mode.
• Subscriptions start on first observation request and are reused. History is bounded to 128 robot samples, 64 camera descriptors and, per image stream, 64 encoded frames / 32 MiB (whichever limit is reached first). Close releases subscriptions. Reconnect after robot/camera generation or Node identity changes.
• To select a historical alignment policy, use robot.observation(mode="aligned", on_timeout="raise"). Repeated calls belong to the external application's loop; there is no separate iterator API.

obs = robot.observation(timeout=0.01)  # Selection budget only; decoding adds time.
if obs.quality.reused:
    print("Reused after timeout:", obs.quality.reuse_age_s, obs.quality.reasons)
    # The caller decides whether to use the old observation or skip this cycle.
else:
    print("Current observation:", obs.quality.valid)

The fallback retains only one complete wire selection, separate from the user's mutable arrays/state. Every returned observation can be modified independently. No age limit is silently reinterpreted: fallback is explicitly invalid and may remain old during a persistent fault. Connection/identity/generation failures and invalid arguments are never hidden by reuse. This is not a hard real-time deadline and does not send, retry or change any robot actions.

RGBD decode performance

Decoded observations use up to three reusable worker threads per connection. After a decoded observation, newly received frames from its requested streams are predecoded in the background for up to 250 ms after the latest request. There are at most three background jobs and one pending frame per active stream; new arrivals replace pending work. images=False, images="encoded" and robot-only requests stop prefetch interest (already running jobs may finish). JPEG decoding uses simplejpeg directly into RGB arrays, with full-quality DCT and upsampling. Pillow remains the PNG/CMYK JPEG and compatibility path. Depth keeps its lossless LZ4/Zstandard codecs and uint16 values; tightly packed decoded depth can use the decompressor's own writable buffer.

The decoder keeps a short history of decoded buffered frames, bounded to 96 entries / 64 MiB total (including retained compressed payloads). Both selection modes can reuse already decoded frames; latest may encounter more cache misses. Identity matching includes the exact buffered object: a new frame, generation or payload does not reuse another frame's result. Arrays in returned observations remain writable and independent: modifying one does not alter another observation or the cache. Calls to the same decoder are serialized so concurrent consumers cannot accumulate an unbounded worker queue. Selection and quality checks run before looking in the decode cache. A miss waits for the exact selected frame; it never substitutes an older decoded frame. Closing the connection joins workers and releases cached frames. Background decoding reduces call latency; it does not remove transport latency or the aligned mode's lookback. The first call and cache misses can take longer than steady-state reads.

Read-only timing/resource check (does not send robot actions):

python examples/benchmark_observation.py --name "NERO Dual" --hz 60 --count 1800

The report separates startup from call latency, counts repeated samples and deadline overruns, and reports source skew, sample age relative to the alignment target, process CPU usage and Linux RSS. The age metric is relative to T; it does not include network transit or any aligned lookback and is not end-to-end latency.

meta_info contains the capability manifest, robot description when available, camera configuration and current calibration bundle/reference. Connection loads it, and observation delivery refreshes it when newer configuration/calibration versions are detected. Property access returns a copy and performs no I/O. Use an observation's calibration reference to identify historical calibration; the metadata cache describes current configuration, not a historical frame.

Fire-and-forget actions

with connect(name="NERO Dual") as robot:
    submission = robot.send_action({"left": {"joints": joint_targets}})
    print(submission)  # Local enqueue only; Node arbitrates on receipt.
    obs = robot.observation()
    print(obs.control.feedback)  # Owner, rejection and per-part execution status.
    # Keep observing until the target completes. Closing here cancels it.

• connect() registers a live session without occupying the robot. No explicit acquire/release API is exposed. The first valid action when idle atomically claims whole-robot control. Other clients receive CONTROL_BUSY in feedback; their commands are not saved or replayed.
• Node releases only after 2 seconds without an accepted action AND all latest targets have reached, followed by fresh hold confirmation. A one-shot target may take longer than 2 seconds. Heartbeats only detect lost connections.
• stop() cancels the current owner's targets and releases after confirmed hold. New commands during that barrier are rejected; submit a fresh action after updated feedback. close() disconnects and stops only that session's targets.
• send_action() and stop() never query or publish on the caller thread. They only normalize/copy bounded target data and enqueue it. The background writer publishes once using Zenoh DROP congestion control. submitted=True means local enqueue only, not delivery/acceptance/execution. Asynchronous transport errors appear in obs.control.local_reason; motion is never retried.
• A locally unavailable session returns ActionSubmission(submitted=False, reason=...). Invalid arguments still raise a programming error. Calls do not exit the process.
• Since 0.1.10: the SDK retries brief heartbeat failures within the existing session lifetime. After expiry or a publish failure, it automatically re-registers with the same Node instance and robot generation; the same Robot object can send new actions once communication recovers. Old pending actions are discarded and interrupted trajectories are not resumed. Calls made while unavailable return submitted=False, without queuing for later. Node restart or robot replacement still requires reconnecting. No extra public heartbeat or session-management calls are needed.
• At most one pending target per arm/gripper and one stop are retained. New targets replace only the same scope (joint and pose share an arm scope). Superseded parts may never be published. The original action ID identifies any surviving parts. This is not an atomic multi-arm transaction. There is no hidden periodic resend.
• Targets are absolute. Pose uses {position, orientation, frame} with a unit xyzw quaternion; {position, rpy, frame} is also supported. Metres/radians and declared arm-base frames are used, and quaternion input is converted to the existing RPY wire fields. Joint/gripper field order and units come from metadata.
• Stop and action share a sequence; stop clears pending old targets. Later targets carry the stop boundary so Node can handle reordered messages. Node must reject old grants, pre-stop commands and stale IK results. These checks run in the Node control manager.
• Internal session fences (control_id) rotate for all registered clients on handover. Queued commands from the old round are discarded, never relabeled. Session expiry requests hold; an unconfirmed hold keeps the safety gate locked.
• Node has bounded execution protection: 30 s per unresolved latest target and 10 s without measured progress, independently of the 2 s idle release rule. These are Node defaults, not per-call SDK arguments. Software hold/stop is not a hardware emergency-stop guarantee.
• obs.control contains local readiness/reason plus the latest per-session Node feedback and its local monotonic receive age. Feedback is a separate stream, copied when the observation is returned, NOT aligned to the sensor selection T. Missing feedback remains None; old feedback is identifiable by control_id and age. It includes per-part action IDs/targets/statuses and the latest rejection.
• Observation reads persistent subscriptions/local caches, not a request per call. Initial discovery/metadata and subscription setup still happen, and selection can wait for samples or decoding; no per-observation request/reply is added.

See control contract for Node integration requirements and complete demo for the external control loop. The loop, policy, pacing, and recording are owned by the external application.

Migration to the small public API

Historical calibration-by-ID is no longer exposed. If recording historical data, retain the calibration bundle in your application and match its bundle ID to the observation reference; do not assume current metadata describes an old frame. No public compatibility aliases are kept. The wire namespace remains lekit/v1; actions require the matching implicit-control Node implementation advertising control_protocol_version: 2. The namespace alone does not guarantee compatibility.

Build and release

From packages/lekit_robot:

python -m pip install build twine
python -m build
python -m twine check dist/*
python -m twine upload --repository testpypi dist/*
# After testing the installed wheel in a clean environment:
python -m twine upload dist/*

Publishing needs a PyPI account and an API token or configured Trusted Publisher. Never store credentials in source files. Version 0.1.0 is the initial release; subsequent releases must use a new version number.

License: Apache-2.0, matching the repository's existing standalone package convention.

A build/test and manual publishing workflow is provided at .github/workflows/lekit-robot-release.yml. It installs the built wheel in an isolated environment before testing. Pushes and pull requests only build/test. After the workflow is committed to the default branch, configure a PyPI pending Trusted Publisher for a new project:

• Project: lekit-robot
• Owner: sorelferris
• Repository: lekit
• Workflow: lekit-robot-release.yml
• Environment: pypi (or testpypi on TestPyPI)

Then run the workflow manually, selecting testpypi first, and pypi for the release. Configure the two registries separately. This avoids storing a long-lived PyPI token. See https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/.

Node control behavior

The device process owns a bounded latest-target scheduler and the same ownership gate used by the workbench, teleop and calibration. control_id identifies a session admission fence; step orders submissions and stop_step excludes older actions. A single target persists while the connection remains alive and execution remains healthy; no continuous action-update deadline is imposed. Release or heartbeat loss requests hold before another controller can acquire. Actions during the stop barrier are rejected; submit fresh actions after hold confirmation. Feedback is available through obs.control.feedback, including per-part states and errors. See Control protocol for confirmation limits.

The old external control channels were removed in 0.1.9; 0.1.10 preserves that public API and adds automatic session recovery. Validation uses simulated device execution and real local Zenoh transport; real robot motion and stop latency require separate on-site acceptance before claiming hardware control performance.