trajlens

The quality and synthesis layer for the open robot-learning data ecosystem.

ruff for robot data — lint, fix, and generate clean LeRobotDataset datasets.

Status

v0.3.0, under active development.

lint is implemented and audited against the public Hub (see Real-world audit below). fix (repair engine: timestamp drift, stats recomputation, episode reindexing, task-index repair, video/info.json fps sync, orphan-shard reporting) and web (read-only local dashboard, now with a per-episode findings view) both ship, with lint --share for redacted issue-report sharing.

Install

pip install trajlens

For dev work:

git clone https://github.com/<your-username>/trajlens
cd trajlens
uv venv
source .venv/bin/activate
uv pip install -e ".[dev,hub,web]"

The [hub] extra pulls in huggingface_hub; it's only required to lint datasets by Hub repo id rather than local path. The [web] extra pulls in FastAPI + uvicorn; it's only required for trajlens web.

Usage

trajlens lint <path-or-org/dataset>          # human-readable terminal report
trajlens lint <path-or-org/dataset> --json   # machine-readable JSON report
trajlens lint <path-or-org/dataset> --report out.html
trajlens lint <path-or-org/dataset> --sarif out.sarif   # SARIF 2.1.0, for CI annotations
trajlens lint <path-or-org/dataset> --deep   # also decode video and verify per-frame stats
trajlens lint <path-or-org/dataset> --share  # redacted summary, for pasting into a GitHub issue
trajlens lint <path-or-org/dataset> --share --share-out out.json  # write it to a file instead

Exit codes follow lint-tool convention: 0 = clean, 1 = WARN present, 2 = FAIL or load ERROR — so trajlens lint composes directly into CI gates.

By default, checks that require materializing a lot of data over the network (full video decode, per-frame stats reconciliation) are skipped for Hub datasets and reported as INFO/skipped rather than run. Pass --deep to force them; expect this to be significantly slower and to fetch the full dataset.

The --json/--report output also includes a per-episode view: the three checks with independent per-episode signal (STRUCTURAL.METADATA_DATA_AGREEMENT, TEMPORAL.TIMESTAMP_MONOTONIC, STATISTICAL.PER_EPISODE_STATS_MATCH) attach a per_episode breakdown to their result, and the report surfaces the worst-5 episodes ranked by trust contribution. --share never includes this free text (see below) — only counts.

--share is a redacted single-file JSON summary safe to paste publicly: trajlens version, trust score, grade, format version, per-check finding counts, the worst-5 episode list (indices and counts only), and a dataset_ref (the Hub repo id for Hub datasets, or the local directory's basename only — never any parent path component). It deliberately omits every check's free-text message/details/per_episode fields, since those can embed local filesystem paths.

Repairing issues: trajlens fix

trajlens fix <local-path>                       # dry-run: preview the diff, write nothing
trajlens fix <local-path> --apply --out <path>  # write a repaired copy
trajlens fix <local-path> --json                # machine-readable dry-run/apply report
trajlens fix <local-path> --only REPAIR.TASK_INDEX_REPAIR       # run only these fixer(s)
trajlens fix <local-path> --except REPAIR.ORPHAN_SHARD_REPORT   # run all applicable fixers except these
trajlens fix <local-path> --apply --out <path> --quarantine     # orphan_shard_report: move orphans instead of just reporting

fix lints the dataset, selects whichever fixers apply to the findings, and runs them in a fixed order. Six fixers exist: REPAIR.TIMESTAMP_DEDRIFT, REPAIR.STATS_RECOMPUTE, REPAIR.EPISODE_REINDEX, REPAIR.TASK_INDEX_REPAIR, REPAIR.VIDEO_METADATA_SYNC, and REPAIR.ORPHAN_SHARD_REPORT. It's copy-on-write — the source is never mutated — and dry-run by default. Only local datasets can be repaired: a Hub ref's data/video shards are streamed on demand and never fully present on disk to copy, so fix refuses Hub refs with a clean error rather than silently producing an incomplete repair.

--only/--except take comma-separated fixer ids and are validated against the known fixer set before any work starts; an unknown id, or the same id in both flags, exits 2 with a message naming the offending id(s). --only also bypasses the normal WARN+ selection threshold, which is the only way to reach REPAIR.VIDEO_METADATA_SYNC today — its target check, VIDEO.RESOLUTION_FPS_MATCH, is catalog-only and not yet implemented as a standalone check, so it never fires on its own. --quarantine only affects REPAIR.ORPHAN_SHARD_REPORT: without it, orphan shards are reported but never moved; with it, apply() relocates them to <output>/.trajlens-quarantine/ and writes a manifest.

Exit codes: 0 = nothing to fix, 1 = fixes proposed or applied, 2 = could not fix (load failure, invalid usage, or a fixer's refusal because the underlying data has no consistent repair).

Viewing a report: trajlens web

trajlens web <path-or-org/dataset>

Lints the dataset once and serves a read-only local dashboard over the result (binds to 127.0.0.1 only, no flag to widen the bind). It's a thin FastAPI shell over the same report the terminal/JSON renderers use — no separate lint logic, no writes, no route that accepts a path/ref/dataset id from the browser.

Found something?

Run trajlens lint <path-or-org/dataset> --share and paste the output into one of the issue forms below. False positives are the most valuable report you can file — if a check flagged something that isn't actually wrong, tell us. There's also a form for bugs and one for new corruption classes trajlens doesn't check for yet.

Architecture

graph TD
  subgraph Interfaces
    CLI[CLI - typer]
    WEB[Web dashboard - FastAPI + static HTML/JS, optional]
    SDK[Python SDK / import]
  end

  subgraph Core
    LOADER[Dataset Source Layer<br/>local + Hub, version-aware]
    MODEL[Canonical Dataset Model<br/>typed in-memory view]
    REGISTRY[Check Registry<br/>pluggable rules]
    ENGINE[Check Engine<br/>runs checks, bounded]
    REPORT[Report Builder<br/>terminal / json / html / sarif]
    REPAIR[Repair Engine<br/>dry-run, diff, opt-in]
  end

  subgraph Synthesis [Pillar 3, later]
    SIMBK[Sim Backend Protocol<br/>MuJoCo default]
    AUG[Trajectory Augmenter<br/>MimicGen-style]
    DR[Domain Randomizer]
    WRITER[LeRobotDataset Writer]
  end

  CLI --> LOADER
  WEB --> LOADER
  SDK --> LOADER
  LOADER --> MODEL
  MODEL --> ENGINE
  REGISTRY --> ENGINE
  ENGINE --> REPORT
  MODEL --> REPAIR
  REPAIR --> WRITER
  SIMBK --> AUG --> DR --> WRITER
  WRITER --> MODEL
  REPORT --> WEB
  HUB[(Hugging Face Hub)] <--> LOADER
  HUB <--> WRITER

What it checks

trajlens validates a LeRobotDataset (v2.0, v2.1, or v3.0) against its own declared metadata, independent of any particular consumer's assumptions. Checks are grouped by category and run as a check engine over each dataset:

Every check's full result — message, severity, and structured details — is included in the JSON/HTML/SARIF report; the table above is the summary.

Real-world audit of the Hub

scripts/audit_hub.py runs trajlens lint --json against a random sample of public Hub datasets tagged lerobot, each in an isolated subprocess with a 60s timeout, and aggregates the results. It's how this project validates itself against the actual long tail of community datasets rather than only its own fixtures.

A 100-dataset run (2026-06-24) produced:

These figures are from a single 100-dataset random sample (raw results: see the v0.1.0 release assets); audit_hub.py samples a fresh random subset of lerobot-tagged Hub datasets on each run, so rerunning it will produce a similarly-shaped but not identical distribution.

Of the 47 load-time ERRORs, none are trajlens bugs: about half (24) are the documented v0.1 limitation that v2.x Hub datasets can't be lazily streamed (shard paths are implicit and require a local filesystem to glob), and the rest are dead/mistagged Hub references, repos that aren't actually LeRobotDatasets (no meta/ directory at the repo root), or genuinely malformed meta/info.json (wrong dtype, missing required fields) on the dataset's side.

TIMEOUTs were investigated as a possible performance bug rather than accepted as an inherent network ceiling: profiling two small, previously-timing-out datasets (abdul004/so101_multi_task_v1, 125 episodes; Elvinky/pick_green_block_into_box, 102 episodes) found that loading a dataset's metadata over Hub HTTP was issuing dozens of small, separately-latency-bound reads per Parquet shard, and downloading the meta/ file tree one file at a time. Fixing both (single whole-shard fetch instead of scattered reads; parallelized meta/ download) brought those two datasets from 60s+ timeouts down to 33s and 11s respectively, and cut the audit's overall TIMEOUT count and mean per-dataset duration by roughly a third in before/after sampling. The remaining TIMEOUTs are concentrated in genuinely large multi-thousand-episode shards, where 60s is a real infra ceiling rather than a fixable inefficiency.

Launch audit findings

Of the 81 datasets that reached a grade (excluding ERROR/TIMEOUT, where no check ever ran), two known upstream lerobot bugs accounted for a meaningful share of the failures:

audit_hub.py resamples a fresh random subset of lerobot-tagged Hub datasets on every run, so these are not a fixed, reproducible distribution — rerunning the audit will not return the same percentages, only a similarly-shaped one. Raw per-dataset results behind these specific numbers are attached to the v0.1.0 GitHub release as audit_results_100.json and audit_summary_100.txt.

Performance note: Hub vs. local

Linting a 100-episode dataset locally takes under 30 seconds.

Linting a Hub dataset directly (trajlens lint org/dataset) streams metadata and data shards over HTTP. It will inherently be slower than a local copy — typically under a minute for small-to-medium datasets, more for very large ones — because of unavoidable network round trips. For repeated linting, downloading the dataset locally first is still faster.

License

Apache-2.0