dexos_lite

Command-line client for Dexos lite dataset upload and robot inference.

Version 0.1.0 requires Python >= 3.10 (megfile 5.0.17 requirement).

Installation

Install from PyPI:

pip3 install dexos-lite

Hardware-specific SDKs are loaded only when a robot controller starts. Install the SDK required by your robot before running robot inference.

Common Options

--api-key can be passed as a command option or provided through an environment variable.

export DEXOS_API_KEY=ak_xxx

Upload A Dataset

dexos-lite dataset upload /data/yellow_pepper \
  --api-key ak_xxx \
  --robot-type mirror \
  --dataset-name yellow_pepper_dataset

Dataset uploads should use the lerobot_v3 format. For the concrete layout, see the data format section.

Optional flags:

dexos-lite dataset upload /data/yellow_pepper \
  --robot-type mirror \
  --dataset-name yellow_pepper_dataset \
  --dataset-format lerobot_v3 \
  --description "yellow pepper pick dataset"

The upload command validates an uncompressed LeRobot v3 directory and uses megfile smart_sync to upload its contents directly to OSS with temporary STS credentials. It preserves relative paths, displays progress, retries failed transfers, and automatically refreshes credentials during long multipart uploads. No archive is created. Symbolic links are rejected.

Uploads require dexos-lite >= 0.1.0. --robot-type accepts cobot_magic, agibot_g1, galaxea_r1_lite, mirror, franka, or other. For other, also pass --custom-robot-type my_robot_v2 (1–128 lowercase letters, digits, or underscores). Python upload functions likewise require robot_type and optionally custom_robot_type.

The client includes client_version in preparation, credential renewal, and completion requests. All three require a stable version >= 0.1.0. The server validates the directory layout and size before completion returns PROCESSING; the conversion worker then validates and converts the data. Only ACTIVE datasets with robot metadata and a valid conversion can be selected for training.

Upload destinations are assigned by the server:

• Built-in robot: datasets/{user_id}/{dataset_id}/{robot_type}/
• Custom robot: datasets/{user_id}/{dataset_id}/other/{custom_robot_type}/

The directory root is uploaded without an extra local folder name. file_size means the total size of source files. The SDK uses a unique in-memory megfile profile and cleans it up after transfer; it does not overwrite your default AWS credentials. The server's long-term AK/SK never reaches the SDK. Credential renewal cannot change the assigned directory. Existing STS tokens remain valid until expiry even after completion.

The low-level prepare_upload method no longer accepts filename or returns a presigned PUT URL. Use the high-level upload_dataset function for validation, directory transfer, automatic renewal, and completion.

Local LeRobot Validation

Upload validation uses the standalone lerobot_validator.py module. Client uploads use fast validation by default. Set full_check=True when every source frame and complete video decoding are required. The conversion worker uses episode_sample_size=10 as described below.

Check	Fast (default)	Full
Robot type, custom identifier, names and declared shapes	Yes	Yes
Tasks, episode metadata, minimum length and file references	Yes	Yes
Required parquet columns, episode/task identifiers and row totals	Yes	Yes
Every state/action value and actual vector dimension	No	Yes
Frame order and timestamps	No	Yes
Video dimensions, decoded frame count and corruption; FPS differences warn	No	Yes

Fast mode reads only episode_index and task_index from data parquet files, groups each file once, and caches episode summaries. Its cost still grows with the number of rows and files, but it does not read vector columns or decode videos. Existing but damaged videos or malformed vector contents may pass fast validation; the worker therefore runs the documented episode sample plus a full output DataLoader scan.

Full mode requires ffmpeg and ffprobe on PATH (for example, brew install ffmpeg on macOS or apt-get install ffmpeg on Debian/Ubuntu). Fast mode does not require either tool.

The module can be copied to another project without the rest of this SDK. It requires pandas and pyarrow; it does not import DexData, PyTorch, OSS, or robot hardware packages.

# If lerobot_validator.py was copied into your project:
from lerobot_validator import validate_lerobot_dataset

report = validate_lerobot_dataset("/data/yellow_pepper")
print(report["mode"], report["episodes_checked"], report["total_frames"])

# After upload, explicitly enable all source checks on the server:
report = validate_lerobot_dataset("/data/yellow_pepper", full_check=True)

When using the SDK package, import from dexos_lite.lerobot_validator instead. The existing validate_lerobot_dataset_root(Path(...)) entry point imports this module and returns (total_frames, task_indices). It also accepts the keyword argument full_check=True. The adapter maps errors to the SDK's DatasetFormatError, DatasetReadError, and DatasetDependencyError types. validate_dataset_path(..., strict=True) uses fast validation; its default non-strict mode only checks basic file properties. The upload flow enables strict validation, which is independent of the standalone validator's full_check option.

You can also run the copied file directly:

python -m pip install "pandas>=2.0" "pyarrow>=14"
python lerobot_validator.py /data/yellow_pepper
python lerobot_validator.py /data/yellow_pepper --full-check

It prints one JSON result and exits with 0 on success or 1 on validation failure (invalid command-line arguments exit with 2). The import API returns a report with status, profile, mode, skipped_checks, episodes_checked, total_frames, videos_referenced, frames_checked, videos_checked, task_indices, fps, robot_type, custom_robot_type, and cameras. Fast reports have zero frames_checked and videos_checked: these counters refer to full content checks, not identifier checks. total_frames is the verified row total. Fast reports list frame_contents and video_contents under skipped_checks; full reports have an empty list. The report profile is lerobot-v3-source-v2. It raises ValidationError, ValidationReadError, or ValidationDependencyError on failure; input files are never changed.

The full source rules come from the independent dataset validation service. The table above identifies which checks fast mode defers:

• LeRobot v3 metadata, positive finite FPS, nonempty robot type, declared vector dimensions, and at least one RGB video camera.
• Tasks with nonempty text in the task column or DataFrame index, consistent task counts/indices, and one task per episode.
• Unique episode indices, exact episode/frame totals, and at least max(3, int(fps)) frames per episode to match the current converter's supported input.
• Every frame in order from index zero, finite nonnegative strictly increasing timestamps, and finite state/action values matching each declared dimension. Timestamps need not equal frame_index / fps.
• Existing references inside the dataset root and consistent episode video intervals. Extra data Parquet files not referenced by metadata produce a warning and are not read or counted. Unregistered or duplicate rows inside referenced Parquet files still fail.
• Full source video decoding, matching dimensions, and enough actual frames for every referenced interval. Declared versus actual FPS differences only warn (both values are logged). Shared video shards may contain more frames than one episode.

Robot Type and Feature Names

The following is a project naming convention, not a mandatory LeRobot standard. Upload parameters and meta/info.json use the same DatasetRobotType enum: cobot_magic, agibot_g1, galaxea_r1_lite, mirror, franka, or other. Values must match exactly. Missing/blank values, legacy aliases such as aloha or open_pdd_w1, uppercase variants, hyphens, and surrounding spaces are rejected.

When robot_type is other, custom_robot_type is required in meta/info.json and in the upload parameters. It is trimmed and must contain 1–128 lowercase letters, digits, or underscores, matching the existing upload contract. Missing, null, blank, and non-string values are rejected. Built-in robot types must not supply a non-null custom_robot_type.

For example, custom robot metadata includes:

{"robot_type": "other", "custom_robot_type": "my_robot_v2"}

The five built-in types skip only the names rules below. Shape, per-row dimensions, finite numbers, file completeness, and all other validation remain mandatory. other always requires names validation, even if custom_robot_type happens to be franka or another built-in identifier. No legacy alias mapping or name inference is used.

For other robots, both features["observation.state"]["names"] and features["action"]["names"] must be nonempty lists of nonempty strings, with no duplicates and exactly shape[0] entries. Null, missing fields, and reference strings such as "observation.state" are rejected. State and action names must be identical and exactly match one of these ordered layouts:

1. Single-arm joints: arm_joint_1 through arm_joint_N, then gripper, for 1 <= N <= 7.
2. Dual-arm joints: left_joint_1 through left_joint_N, left_gripper, then right_joint_1 through right_joint_N, right_gripper. Both arms must have the same number of joints, from 1 to 7.

Names are case-sensitive and are not trimmed or normalized. Joint numbers must start at 1 without gaps; order is mandatory. Single- and dual-arm names cannot be mixed. End-effector position and rotation names (including rotvec, Euler, and quaternion), base/torso/hand joints, torque, velocity, and any other extra vector dimensions are rejected. Each arm requires one scalar gripper. Other LeRobot features, including images and timestamps, are not subject to these names rules. This validator does not map slots, reorder vectors, pad to 32 dimensions, or generate masks; it only checks the supplied names and data.

This replaces the old fixed 14-dimensional check and first-frame sampling. Pose fields, stats.json, and the global index column are not required by these source rules. Full mode keeps source parquet rows in memory; video decoding runs one file at a time. Fast mode retains episode summaries after processing each file. Full decoding includes unused gaps in referenced source videos, so a damaged gap also fails local validation. Unreferenced source videos are not inspected.

Task text can be stored in tasks.parquet's task column or its text index. If both contain text, they must agree exactly; an invalid task column is never silently ignored. When present, task_index must still match row positions 0..n-1. The SDK does not rewrite input files. The server must normalize the validated texts into the temporary extracted copy's task index before invoking the current converter, then reconcile every output prompt with the original texts. Use the matching server update with this SDK change; the older converter alone ignores the task column and would produce numeric prompts.

Local success is source validation, not server-side publication approval. Archive safety, LeRobot-to-DexData conversion, conversion reconciliation, algorithm DataLoader checks, OSS upload, and _SUCCESS publication remain server-side. It does not certify annotation accuracy, joint semantics, physical safety, or training results.

Generate Robot Config

Generate config.py in the current directory:

dexos-lite robot config --robot-id mirror

Generate a config at a specific path:

dexos-lite robot config \
  --robot-id mirror \
  --path ./configs/mirror_config.py

Overwrite an existing config:

dexos-lite robot config \
  --robot-id mirror \
  --path ./config.py \
  --force

Supported robot_id values:

• mirror
• cobot_magic

Edit the generated config before inference to set controller ports, sensor serial numbers, and robot-specific options. Inference requires at least one camera sensor image. Each image must be a JPEG or PNG data URL. Images larger than 448 * 448 are centered on a black square canvas and resized to 448 * 448 before the request is sent.

Mirror Arm Service

For mirror, start the Mirror arm service before running inference. Use the mirror documentation as the source of truth for exact service options.

conda activate airbot && sleep 1 && airbot_server -i can_left -p 50051
conda activate airbot && sleep 1 && airbot_server -i can_right -p 50053

Run Robot Inference

dexos-lite robot infer \
  --api-key ak_xxx \
  --robot-config ./config.py \
  --prompt "pick up the red cube"

--robot-config accepts path.py or path.py:attribute. If no attribute is provided, robot is used.

robot infer runs for up to 600 seconds by default. Use --time-limit to set a different limit in seconds.

--continue starts inference from the robot's current pose by skipping the initial robot.reset_to_home() call. The CLI still calls robot.start() first to start hardware resources.

When inference ends because of Ctrl+C or the time limit, the CLI asks for a score from 0 to 10 and an optional feedback message, then submits it to service. When inference is interrupted with Ctrl+C, the CLI calls robot.reset_to_home() again before closing the robot.

Use Ctrl+C to stop the inference loop.

Joint-space controller commands use a fixed default interpolation step of 0.02. Set the uppercase DEXOS_INTERPOLATION_STEP environment variable to a positive float to override it, or to 0 to disable interpolation:

export DEXOS_INTERPOLATION_STEP=0

Use A Custom Robot

--robot-config can point to any Python file that exposes a BaseRobot instance or factory. If no attribute is provided, robot is used.

Custom robots must implement action_type, start(), get(), set(actions), reset_to_home(), and close().

get() must return a dict with:

• state: required. JSON-serializable current robot state sent to service as top-level state; for common arm models this is usually a flat list such as [j1, j2, j3, j4, j5, j6, gripper]. Refer to the /api/v1/action API docs for state element ordering.
• sensors: required. Dict of camera name to base64-encoded JPEG or PNG data URL. infer_robot sends this to the service as top-level images. Provide at least one image. Images larger than 448 * 448 are centered on a black square canvas and resized to 448 * 448 before the request is sent. It supports at most three keys: cam_high, cam_left_wrist, and cam_right_wrist.

Each call to set(actions) receives and executes one model action step from an action horizon. The layout and element order of the action match the state returned by get(). The inference API may return multiple steps at once as an action horizon. The CLI passes each step to set() in order and requests the next inference result after the current action horizon has finished.

Continuous CLI inference does not impose an additional action rate limit. Therefore, set() must pace the hardware and return only after the current action has completed safely.

reset_to_home() is called once before inference starts and again when inference is interrupted with Ctrl+C. Custom robots should move the hardware to a safe home pose there; if there is no real home action, still return a JSON-serializable result such as {"ok": True}.

Create custom_robot.py:

from __future__ import annotations

from typing import Any, Dict

from dexos_lite import ActionType
from dexos_lite.robot import BaseRobot


class CustomRobot(BaseRobot):
    @property
    def action_type(self) -> ActionType:
        return ActionType.JOINT

    def start(self) -> None:
        # Initialize SDKs, connect hardware, or start external services here.
        pass

    def get(self, *, sensor_timeout: float = 1.0) -> Dict[str, Any]:
        current_state = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
        # Read cameras within sensor_timeout.
        # sensors requires at least one image and supports at most these three fields.
        sensors = {
            "cam_high": "data:image/jpeg;base64,...",
            # "cam_high": "data:image/png;base64,...",
            # "cam_left_wrist": "data:image/jpeg;base64,...",
            # "cam_right_wrist": "data:image/jpeg;base64,...",
        }
        return {"state": current_state, "sensors": sensors}

    def set(self, actions: Any) -> Dict[str, Any]:
        # Send actions to your robot SDK or hardware service here.
        return {"ok": True}

    def reset_to_home(self) -> Dict[str, Any]:
        # Move the robot to a safe home pose here.
        return {"ok": True}

    def close(self) -> None:
        pass


robot = CustomRobot()

Run inference with it:

dexos-lite robot infer \
  --api-key ak_xxx \
  --robot-config ./custom_robot.py:robot \
  --prompt "pick up the red cube"

You can also expose a factory, for example def build_robot() -> CustomRobot: ..., and pass --robot-config ./custom_robot.py:build_robot.

Conversion worker validation (0.1.0)

The worker pins dexos-lite==0.1.0, matching the upload server's documented minimum version. Publish the reviewed 0.1.0 package containing the sampling API before building the worker image from the package index.

from dexos_lite.lerobot_validator import validate_lerobot_source
source = validate_lerobot_source("/data/dataset", episode_sample_size=10)
report = source["report"]

All episode metadata, file references, identifier counts, and task assignments are checked. The validator deterministically selects up to ten episode IDs, evenly spaced over sorted IDs including both endpoints (all when there are at most ten). Every frame and every camera segment in selected episodes is checked. Only selected frame rows are retained in source["episodes"]; that list preserves metadata traversal order so conversion output serials remain aligned. Shared Parquet uses filtered batches and shared video uses segment seeking, which may decode from a preceding keyframe. No complete frame-table cache is retained.

validate_lerobot_dataset(..., episode_sample_size=10) returns only the summary. Existing default fast preflight and explicit full_check=True remain available for callers that need those distinct scopes. The conversion worker compares only the selected episodes, releases their source rows, then runs its DataLoader over every output episode and frame in batches of 64. Video decoding is sampled; DataLoader checks numeric values and media references, not image loading.

Video FPS mismatches do not reject readable data. Metadata interval duration differing from length / info.fps also warns; intervals must still have finite nonnegative starts and an end after the start. Sample and full validation seek to the actual referenced start and decode exactly the required episode frame count. The optional end is informational and never truncates the required frames or causes extra frames from the next episode to be read. Frame-boundary float rounding is tolerated; frames are never duplicated to repair gaps. Corrupt video, an unreadable start or insufficient required frames remain errors. An inaccurate end beyond EOF may warn and pass if every required frame is readable. Full mode additionally decodes the complete shared video, including unreferenced gaps.