TacStack

English | 简体中文

Cross-sensor tactile semantics and runtime for robotics. TacStack keeps raw tactile data intact and layers a unified Observation, model interface and Event contract on top, steadily lowering the cost of bringing up and using different tactile sensors.

Status: Phase 4 done / Phase 5 (first live sensor) in preparation. The offline loop works end to end — dataset → TactileObservation → MCAP / Rerun replay → contact / slip events (builtin or ONNX scoring) → benchmark report. Observation streams export to and replay from LeRobot v3.0 dataset directories (taxel features), riding that ecosystem's tooling directly. Protocol layers for five real sensors are implemented and tested offline (Verified sensor protocols): four serial codecs verified byte-for-byte against their vendor manuals, plus the Meta DIGIT vision-tactile identity / semantics layer. Live adapters land with the hardware (Phase 5). ROS2 integration is not implemented yet; the bundled dataset fixture is synthetic (tests/fixtures/README.md).

Getting started

Requires Python 3.12+ and uv. Install from source for now; no PyPI package yet.

git clone https://github.com/mailes/TacStack.git
cd TacStack
uv sync
uv run tacstack version
uv run tacstack contract-demo
uv run tacstack demo
uv run python examples/contract_demo.py

contract-demo prints one JSON event flagged synthetic: true. It downloads no data and touches no hardware.

The same Observation contract also replays real Open-X-Tactile episodes from a local tar or extracted directory (CI and the example below use only the bundled synthetic fixture):

uv run tacstack dataset list tests/fixtures/open_x_tactile/demo_wipe.tar
uv run tacstack dataset inspect tests/fixtures/open_x_tactile/demo_wipe.tar --task Wipe_Demo --episode 0
uv run tacstack dataset convert tests/fixtures/open_x_tactile/demo_wipe.tar --task Wipe_Demo --episode 0 --out demo.mcap
uv run python examples/replay_open_x_tactile.py

from tacstack import TactileEvent

sample = TactileEvent(
    timestamp_ns=0,
    sensor_id="synthetic",
    kind="contact_begin",
    probability=1.0,
    model_id="contract-example",
    latency_ms=0.0,
)

Verified sensor protocols

Five real sensor protocol layers ship hardware-free. Four are pure codec layers — frame building and parsing, checksums, register decoding — verified byte-for-byte against the worked example frames in their vendor manuals. The fifth is the Meta DIGIT vision-tactile layer: device identity, raw-frame to observation mapping and reference-frame contact / slip tracking. All are pinned by unit tests with no hardware required. Live capture adapters plus live record / replay / inference are the Phase 5 deliverables once the hardware arrives.

Request-frame checksums are pinned by the manuals' example frames. Where a rule cannot be recovered from documentation alone (the response-frame CRC on PX6D / PX3Q), parsing stays lenient and the open gap is recorded in the module docstring until captured hardware traffic settles it. Selection criteria and buy-time caveats live in docs/plans/sensor-procurement.zh-CN.md.

Project layout

src/tacstack/
  core/           # data structures, basic validation, debug serialization
  adapters/       # base protocol; open_x_tactile + mcap replay + real_sensor codecs + DIGIT semantics implemented
  runtime/        # model protocol; buffer / ONNX backend reserved
  models/         # contact / slip reserved
  integrations/   # MCAP export + Rerun replay + LeRobot v3.0 dataset export implemented; ROS2 reserved
  benchmark/      # evaluation tooling reserved
  cli/            # version / contract-demo / dataset / replay / annotate / model run / benchmark
examples/         # runnable contract_demo and replay_open_x_tactile
tests/            # unit / integration / fixtures
docs/             # architecture, interfaces, roadmap, ADR (Chinese for now)
scripts/          # development tools (fixture generator)
.github/workflows/ci.yml

Development checks

uv run ruff check .
uv run ruff format --check .
uv run mypy src
uv run pytest -m "not hardware"
uv build
uv run pre-commit install

Next steps

1. Bring up the first real sensor end to end — live adapter, live record / replay / inference, 30-minute stability (Phase 5).
2. Docs, external Quickstart and the v0.1 release (Phase 6).

Runnable walkthroughs: Tutorial 1 - offline pipeline and Tutorial 2 - visualize & annotate. See also the roadmap, architecture, concepts, adapter guide, runtime, benchmark and the contribution guide. Most of these docs are Chinese for now; English translations will follow before the v0.1 release.

Principles and license

Raw-first · Semantics-first · Edge-first · Integration-first. Core depends on neither ROS2, cloud services nor training frameworks; reuse the existing storage and visualization ecosystems.

Apache-2.0, see LICENSE. Dataset, model and third-party SDK licenses must be verified separately.