lekit-robot

Small Python SDK for a running Lekit Robot Node. Discover a robot by name, read joint/pose feedback and RGB/depth images, and send absolute motion targets. The client does not install LeRobot, Torch, OpenCV, robot drivers, or camera drivers. Python 3.10+ is required.

Install

Install or upgrade:

pip install -U lekit-robot

From this repository:

pip install ./packages/lekit_robot

Robot Node must already be running on the robot computer. Clients need network access to its Zenoh peer; the browser's HTTP port 8080 is not the SDK endpoint. LAN multicast discovery is the default. If discovery is blocked, configure both Node (lekit-robot --zenoh-router tcp/ROUTER_IP:7447) and client (connect(name="NERO Dual", router="tcp/ROUTER_IP:7447")) to use an existing Zenoh router. This package installs the client CLI, not the Robot Node server. In a full lekit installation, lekit-robot-node starts the server; the former lekit-robot --host ... --port ... invocation remains compatible.

Continuous observation test

lekit-robot nero-dual --observation
# Exact advertised name; 30 Hz RGB-only observations over an explicit Zenoh endpoint:
lekit-robot "NERO Dual" --observation --router tcp/192.168.3.241:7447 --hz 30 --mode rgb
# State only, or restrict the selected cameras:
lekit-robot "NERO Dual" --observation --mode robot
lekit-robot "NERO Dual" --observation --camera head_view --camera left_view
# Compare with the previous 50 ms lookback selection:
lekit-robot "NERO Dual" --observation --observation-mode aligned

Run this on the policy/client computer to include its network path. It calls the real observation() API using persistent subscriptions, without acquiring control or sending actions. The Rich panel continuously reports achieved return frequency, call P50/P95/P99, startup, errors, repeated samples, period overruns, estimated source ages at return (latest and P95), source ages at T, skew, decode/cache metrics, CPU and memory. CPU 100% means one core; RSS is available on Linux. Measurement includes diagnostic overhead.

Default: 60 Hz, latest complete samples with timestamp validation, RGB and enabled depth from all cameras, runs until Ctrl+C. --observation-mode latest|aligned selects sample timing independently of --mode (image format/content). Use --hz, --mode rgbd|rgb|encoded|robot, --camera NAME (repeatable), --timeout .2 (selection wait seconds; decoding can extend total call time), --connect-timeout 3, or optional --count 300 for a bounded run. The panel refreshes twice per second; rates/percentiles cover a bounded three-second window. First successful call is shown separately from steady-state latency; counts are cumulative. Ctrl+C/SIGTERM closes the client and retains the final panel.

The positional name first matches the advertised name (case insensitive). If not found, a hyphenated CLI name such as nero-dual retries as nero dual. Ambiguous names are rejected. --obervation is also accepted as a compatibility spelling. python -m lekit_robot ... is equivalent to the console command.

The observation freshness table distinguishes two ages:

• Return age ≈: client UTC at observation() return minus the selected sample's Node capture UTC. It includes transport/decode waiting, plus the 50 ms lookback when using aligned. The adjacent P95 covers successful observations in the last three seconds. Reusing a sample increases its return age.
• Age at T: alignment target T minus sample time in the Node clock domain. In latest, T is the newest known selected-source timeline; in aligned, it includes the lookback. Age at T is not capture-to-return latency.

Return age is an estimate requiring NTP/PTP synchronization between client and Node; this tool does not calibrate clocks. Clock offset affects even positive values. Missing/timezone-naive capture times display —; negative ages display 时钟偏差 and are excluded from percentiles. Cross-host monotonic clocks are never subtracted. Camera ages describe frame groups; robot ages describe Node snapshots, not each underlying CAN message. On observation failures the table is labeled as the last successful observation. Node loss terminates the test; rerun after reconnecting.

The observation test command is available in lekit-robot 0.1.6 and later. Return-age freshness metrics require 0.1.7 or later. The default latest selection and --observation-mode require 0.1.8 or later.

Three primary interfaces

Version 0.1.6 provides observation(), send_action(), and meta_info in the existing package. send_action() requires the matching Robot Node target-v1 extension; there is no silent fallback to the older fire-and-forget stream. Existing public methods remain available.

from lekit_robot import connect

with connect(name="NERO Dual") as robot:
    meta = robot.meta_info                   # cached; no network query here
    obs = robot.observation()                # robot + all configured camera streams
    print(obs.robot.state, obs.quality)
    print(meta.manifest.joint_target.fields) # ordered names, units, bounds
    print(meta.calibration)                  # None if no calibration is available

    # Other selections, using the same persistent subscriptions:
    state = robot.observation(cameras=(), images=False)
    images = robot.observation(cameras=["head_view"], robot=False, depth=False)

Observation contract

• Default: decoded RGB and enabled depth, robot state, all advertised cameras, strict=True. images="encoded" retains compressed frames; images=False returns sample metadata without decoding; depth=False omits depth payloads.
• mode="latest" (default) selects the newest complete sample from each requested source, without a fixed lookback. Selection happens under one buffer lock and validates timestamp skew and age before returning. Camera metadata, RGB and requested depth must belong to the same frame group; if a newer group's payload is missing, the newest older complete group can be used, subject to those checks. T is the newest known timestamp among the selected sources, including robot message preparation timestamps. Excessive skew waits for an update and raises on timeout in strict mode; it never silently disables alignment checks.
• mode="aligned" retains the previous 50 ms lookback from the latest received robot timeline (or selected cameras when robot state is excluded). Each source contributes its newest capture at or before T. Strict aligned mode waits for the selected frame's payload instead of falling back to an older complete frame. Robot timestamps describe the SDK snapshot read, not simultaneous acquisition of every CAN feedback field. Alignment is not hardware synchronization.
• at=T uses seconds in the connected Node monotonic clock, not client time.monotonic() or Unix time. An explicit at always uses causal aligned selection at exactly T, even with the default mode; it adds no lookback. For example, use a prior observation's aligned_at_monotonic_ns / 1e9. Only already buffered history can be selected.
• Defaults: age at T <= 250 ms, source timestamp skew <= 50 ms. Override with max_age={"robot": .1, "camera": .2, "head_view": .15} and max_skew=.03. Local time since the source last arrived also detects a stalled subscription.
• Missing/stale sources, mismatched image payloads, excessive skew, and decoding failures raise ObservationUnavailableError after the bounded wait (decode failures fail immediately). Its .quality.reasons explains the failure; .quality is None if no target could be established. strict=False permits partial/invalid results; always inspect obs.quality.valid in that mode.
• Subscriptions start on first observation request and are reused. History is bounded to 128 robot samples, 64 camera descriptors and, per image stream, 64 encoded frames / 32 MiB (whichever limit is reached first). Close releases subscriptions. Reconnect after robot/camera generation or Node identity changes.
• Legacy get_observation() / observations() retain their existing selection behavior. To restore the previous strict facade behavior use robot.observation(mode="aligned").

RGBD decode performance

Decoded observations use up to three reusable worker threads per connection. After a decoded observation, newly received frames from its requested streams are predecoded in the background for up to 250 ms after the latest request. There are at most three background jobs and one pending frame per active stream; new arrivals replace pending work. images=False, images="encoded" and robot-only requests stop prefetch interest (already running jobs may finish). JPEG decoding uses simplejpeg directly into RGB arrays, with full-quality DCT and upsampling. Pillow remains the PNG/CMYK JPEG and compatibility path. Depth keeps its lossless LZ4/Zstandard codecs and uint16 values; tightly packed decoded depth can use the decompressor's own writable buffer.

The decoder keeps a short history of decoded buffered frames, bounded to 96 entries / 64 MiB total (including retained compressed payloads). Both selection modes can reuse already decoded frames; latest may encounter more cache misses. Identity matching includes the exact buffered object: a new frame, generation or payload does not reuse another frame's result. Arrays in returned observations remain writable and independent: modifying one does not alter another observation or the cache. Calls to the same decoder are serialized so concurrent consumers cannot accumulate an unbounded worker queue. Selection and quality checks run before looking in the decode cache. A miss waits for the exact selected frame; it never substitutes an older decoded frame. Closing the connection joins workers and releases cached frames. Background decoding reduces call latency; it does not remove transport latency or the aligned mode's lookback. The first call and cache misses can take longer than steady-state reads.

Read-only timing/resource check (does not send robot actions):

python examples/benchmark_observation.py --name "NERO Dual" --hz 60 --count 1800

The report separates startup from call latency, counts repeated samples and deadline overruns, and reports source skew, sample age relative to the alignment target, process CPU usage and Linux RSS. The age metric is relative to T; it does not include network transit or any aligned lookback and is not end-to-end latency.

meta_info contains the capability manifest, robot description when available, camera configuration and current calibration bundle/reference. Connection loads it, and observation delivery refreshes it when newer configuration/calibration versions are detected. Property access returns a copy and performs no I/O. Use an observation's calibration reference to identify historical calibration; the metadata cache describes current configuration, not a historical frame.

Unified actions

# These functions send motion only when called with a connected robot.
def update_left_arm(robot, joint_targets, gripper_m):
    return robot.send_action({
        "left": {"joints": joint_targets, "gripper": gripper_m},
    }, timeout=.1)


def update_right_pose(robot, position_m, rpy_rad):
    return robot.send_action({
        "right": {"pose": {
            "position": position_m, "rpy": rpy_rad, "frame": "right.base",
        }},
    }, timeout=.1)

• Absolute targets only. Joint arrays follow advertised field order; mappings use exact advertised field names. Pose position uses metres and RPY uses radians in the selected arm's base (left.base / right.base, or robot.base for a single robot). Gripper units and limits come from the manifest. Omitted arms/grippers retain their own target until their own expiry. A single arm cannot specify joints and pose together. Node performs Cartesian IK using nero-dual-ik for NERO.
• send_action() is synchronous until Node validation and driver dispatch have returned, then returns an ActionReceipt(status="accepted"). It never waits for physical arrival. This is not an async def API. A receipt's accepted_at_unix_ns is a Node wall-clock timestamp.
• First send acquires control; a background thread renews the lease. Subsequent sends reuse it. Calls are serialized, one request in flight per client; there is no action chunk queue. A 60 Hz loop requires acceptance latency below its 16.7 ms budget; no real-hardware 60 Hz guarantee has been established.
• connect(target_ttl=.1) requests a hold when an arm/gripper receives no update for 100 ms (range .05–2 s). Updates to another arm/gripper do not extend this deadline. target_ttl=None explicitly leaves targets active without a target timeout; abrupt client loss does not guarantee a hold in this mode. Graceful control release/close requests a hold while the lease is still valid.
• New absolute targets supersede the corresponding current targets. All parts are validated before dispatch, but physical writes across arms are not atomic. Lost replies or partial driver errors raise ActionOutcomeUnknownError with .request_id; do not retry automatically. A rejected request raises ActionRejectedError with .code. Acceptance is never arrival or stop confirmation.
• Node also checks request age using Unix clocks, so controller and Node clocks must be synchronized. A hold is subject to fresh feedback, generation and ownership checks; it is not a safety-rated emergency stop.

Gradual migration

observations(), task lifecycle methods and historical get_calibration(bundle_id) remain available. Existing stream_joints() / set_target(mode="stream") still return submitted, not Node acceptance. Existing observation methods keep their best-effort behavior. There is no dataset writer, episode model or recording scheduler in this refactor.

Legacy interfaces

Observe (does not acquire control)

from lekit_robot import connect

with connect(name="NERO Dual", timeout=5) as robot:
    obs = robot.get_observation(images=True, timeout=5)
    print(obs.robot.status, obs.robot.state)
    camera = obs.camera(name="head_view")
    if camera.color is not None:
        print(camera.color.shape)  # H x W x 3, RGB uint8
    print(robot.descriptor.joint_fields)

images=False omits images. robot.observations(images=False, timeout=5) aligns independently published Robot and Camera samples with bounded latest-only buffers. get_observation() is a one-shot convenience over the same stream alignment path; the Node has no separate snapshot publication or observation query protocol. For alignment time T, each source contributes its newest sample captured no later than T; network arrival time is never used as capture time. Camera samples remain independent, so an observation is not a hardware-synchronized frame set. Check each sample's capture time and monotonic timestamp. Depth is a uint16 array in the server's camera units, not automatically metres. The Node's orbbec plugin outputs millimetres (0 means invalid), with depth aligned to RGB by default. Read it with obs.camera(name="head_view").depth; convert to metres with depth.astype("float32") / 1000. The Orbbec SDK is only needed on the server, not on this client. On the wire, aligned depth is always losslessly compressed using the camera's configured LZ4 or Zstandard codec. The SDK transparently decompresses it for images=True; use images="encoded" to retain the compressed payload and call camera.decode_depth() only when needed.

Each observation also carries a small calibration reference that is timestamped with the robot state. Fetch the complete, versioned bundle only when the reference changes, then keep it in the recorder or application cache:

with connect(name="NERO Dual", timeout=5) as robot:
    obs = robot.get_observation(images="encoded", timeout=5)
    ref = obs.calibration
    if ref is not None and ref.bundle_id is not None:
        bundle = robot.get_calibration(ref.bundle_id, timeout=5)
        camera = next(binding for binding in bundle.camera_bindings
                      if binding.camera_name == "head_view")
        print(ref.state, camera.intrinsics, camera.sensor_frame)
        print(bundle.edges)  # Check source_frame, target_frame, status and transform.

robot.get_calibration() returns the current bundle. Passing a bundle_id returns that immutable historical bundle, so recorded observations remain reproducible after a later recalibration. A reference state of degraded or invalid means the bundle is available but one or more transform edges no longer match the active camera bindings; downstream code decides whether those edges are usable. For the geometry of a delivered image, read obs.camera(name="head_view").geometry. If no validated bundle is active, get_calibration() raises CalibrationNotFoundError.

Control

Control requires enabled motors, allow_control=True, fresh feedback, normal mode, and no emergency stop. Close Teleop before acquiring external control. The context manager holds and renews an exclusive control lease and releases it on exit. Multiple clients may observe; only one controller may own the lease.

The following function sends a real motion command when called:

from lekit_robot import connect

def move_left_joint3(delta_rad):
    with connect(name="NERO Dual") as robot:
        with robot.control():
            obs = robot.get_observation(images=False)
            state = obs.robot.state
            if obs.robot.status != "connected" or not state or not state.get("left.feedback_fresh"):
                raise RuntimeError("Fresh left arm feedback is required")
            joints = [state[f"left.joint_{i}"] for i in range(1, 8)]
            joints[2] += delta_rad
            return robot.set_joints(joints, side="left")

For NERO Dual, left is can1 and right is can0. Joint/rotation units are radians; position and gripper width units are metres. Pose fields represent the flange in that arm's base frame. For other robots use the plugin-declared field semantics. side selects only that arm; the other arm receives no target. Without side, provide the complete advertised target (14 joint values or 12 pose values for NERO). A scalar gripper width requires side on a dual-arm robot. There is no implicit broadcast of one pose or gripper value to both arms.

NERO uses nero-dual-ik for Cartesian targets without an SDK or firmware IK fallback. stream_joints() and set_target(mode="stream") use the Node's latest-only stream channel and arm its loss-of-stream watchdog. Their immediate receipt has status="submitted"; it does not say whether the asynchronous device validation accepted the target. Streaming does not guarantee a hard realtime servo or a straight path. Unsupported modes, partial targets and out-of-range values are rejected.

A receipt acknowledges command submission, not arrival or successful IK. Continue observing measured state and alarms. ActionOutcomeUnknownError means a command may already have executed: do not retry it automatically. A control lease is ownership arbitration, not a hardware emergency stop or a stop-on-disconnect guarantee. Use the Robot Node emergency stop or physical stop as appropriate.

For a joint target that needs confirmed arrival, use a device-owned motion task:

import time

from lekit_robot import connect

with connect(name="NERO Dual", timeout=5) as robot:
    task = robot.start_joint_task({"left.joint_3.pos": 0.2}, timeout_s=15)
    while not task.done:
        time.sleep(0.1)
        task = robot.get_motion_task(task.task_id)
    if task.state != "succeeded":
        raise RuntimeError(f"Motion task ended as {task.state}: {task.error}")

A task acquires its own exclusive control lease; do not enter robot.control() around it. start_joint_task() returns an accepted task, not a completion guarantee. cancel_motion_task(id) requests a stop; poll until cancelled with stop_confirmed=True or inspect failed. If the start reply is lost, MotionTaskOutcomeUnknownError.task_id identifies the task to query before retrying. The Node currently supports joint tasks only.

Compatibility

The wire protocol remains lekit/v1. New clients opt into the gripper manifest extension; old clients continue to receive the original manifest shape. An old Node can still supply observations and existing targets, but gripper commands need an updated Node; unsupported grippers raise UnsupportedCommandError locally. Per-arm commands also require the updated Node validation. Update client and Node together for complete NERO Dual support. Reconnect after replacing the Node or its robot configuration; the SDK never silently rebinds an existing control session.

Existing applications may keep from lekit.robot import connect when the main project is installed: those imports re-export this package's implementation.

Build and release

From packages/lekit_robot:

python -m pip install build twine
python -m build
python -m twine check dist/*
python -m twine upload --repository testpypi dist/*
# After testing the installed wheel in a clean environment:
python -m twine upload dist/*

Publishing needs a PyPI account and an API token or configured Trusted Publisher. Never store credentials in source files. Version 0.1.0 is the initial release; subsequent releases must use a new version number.

License: Apache-2.0, matching the repository's existing standalone package convention.

A build/test and manual publishing workflow is provided at .github/workflows/lekit-robot-release.yml. It installs the built wheel in an isolated environment before testing. Pushes and pull requests only build/test. After the workflow is committed to the default branch, configure a PyPI pending Trusted Publisher for a new project:

• Project: lekit-robot
• Owner: sorelferris
• Repository: lekit
• Workflow: lekit-robot-release.yml
• Environment: pypi (or testpypi on TestPyPI)

Then run the workflow manually, selecting testpypi first, and pypi for the release. Configure the two registries separately. This avoids storing a long-lived PyPI token. See https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/.