PyRoboReplay

Forensic debugging and mission-replay platform for autonomous robots: replay ROS 2 missions, run causal / root-cause analysis on failures, and perform multispectral (RGB + thermal) forensics. Rust core with Python bindings.

What it is

PyRoboReplay ingests recorded robot missions (ROS 2 bags) into a unified timeline and answers why and what-if questions rather than only where and what: which events preceded a failure, what the perception stack missed, and what the robot should have detected. The heavy lifting is a Rust core (cdylib + rlib) built with PyO3, exposed to Python through maturin. A CLI (replay, analyze, compare, list) and a small Python API are the supported entry points today.

Features

Exposed via CLI / Python (verified, tested):

• Mission replay — interactive timeline scrubber with keyboard controls (replay).
• Reality-gap analysis — probabilistic mismatch scoring, severity classification, evidence aggregation, JSON/text output (analyze --detect-gaps).
• Failure detection and diagnosis — Mission.detect_failures(), Mission.analyze_failure() with hypotheses, confidence, and prioritized actions.
• Mission comparison — side-by-side diff of two bags (compare).
• Topic listing — inspect available topics in a bag (list).
• Real ROS 2 CDR decoding — standard sensor_msgs / nav_msgs / geometry_msgs binary layouts are decoded from real message bytes (src/adapters/cdr.rs); pose confidence is derived from the real position-covariance trace, so localization-loss detection fires on real data.

Implemented in the Rust core, not yet wired to CLI or Python (library API only):

• Causal event graphs, multi-factor causality, incident narratives, evidence-quality scoring.
• Persistent world knowledge (entities, locations, facts across missions) and spatial grounding.
• Multi-mission / longitudinal learning.
• Pluggable detection backends (YOLO / SAM / template fallback).
• Retrospective DINO + SAM open-vocabulary object discovery.
• RGB + thermal multispectral fusion and forensic reconstruction.
• Universal temporal fusion and the Nav2 root-cause inference engine (Phases 14-15).

Who it's for / Use cases

• Warehouse and logistics fleets — reconstruct why a run failed, find missed detections and coverage gaps, compare two strategy revisions.
• Safety and compliance review — produce auditable, saved JSON findings for an incident (analyze --detect-gaps --format json --output report.json).
• Perception R&D — compare perception strategies over matched missions and quantify regressions.
• Field autonomy teams — investigate localization loss, planner oscillation, and sensor disagreement from recorded missions.

What it is not

• Not a real-time monitoring or fleet dashboard. It works on recorded missions, not live streams.
• Most advanced modules (Phases 12-15) are not reachable from the CLI or Python. Use the Rust library directly, or track exposure on the roadmap.
• Phase 11 "PyTerrainMap integration" is not real interop. src/.../pyterrain_bridge.rs is self-contained, unit-tested terrain-modeling logic conceptually inspired by PyTerrainMap; there is no Cargo/pip dependency, no shared serialization format, and no parser for PyTerrainMap's output.
• Not published to crates.io — build the Rust crate from source if you need it directly.
• Not a certified or safety-verified system.

Project status

Beta (Development Status 4). Current source version is 2.10.1; the latest PyPI release is 2.10.0. The CLI and Python API are the tested, supported surface. 843 cargo test --lib unit tests pass (re-verified 2026-09-29), plus 7 real-fixture ROS 2 CDR integration tests. cargo fmt --check is clean. cargo clippy --all-targets --all-features -- -D warnings currently reports ~96 errors, so the CI clippy job runs with continue-on-error and does not gate merges. See ROADMAP_HONEST.md for the implemented/partial/missing breakdown.

Installation

pip install pyroboreplay

# or with uv
uv pip install pyroboreplay

# verify
pyroboreplay --version

Requires Python 3.10+. From source (needs a Rust toolchain):

git clone https://github.com/Mullassery/PyRoboReplay.git
cd pyroboreplay
pip install maturin
maturin develop --release      # install the Python package
# or, for the Rust binary/crate only:
cargo build --release

Quick start

# Interactive timeline scrubber
pyroboreplay replay mission.bag

# Reality-gap analysis, saved as JSON
pyroboreplay analyze mission.bag --detect-gaps --detail --format json --output investigation.json

from pyroboreplay import Mission

mission = Mission.from_ros_bag("warehouse.bag")

failures = mission.detect_failures()
print(f"Failures detected: {len(failures)}")

analysis = mission.analyze_failure(timestamp=1234567890.0)
print(f"Root cause: {analysis.get_primary_hypothesis()}")
print(f"Confidence: {analysis.get_diagnostic_confidence():.0%}")

for action in mission.recommend_actions(timestamp=1234567890.0):
    print(f"[{action.get_priority()}] {action.get_description()}")

Interactive replay keys: Space play/pause, n / → next event, p / ← previous event, ↑ / ↓ speed, ? help, q / Esc quit.

Usage

pyroboreplay compare mission_a.bag mission_b.bag        # side-by-side
pyroboreplay list mission.bag                            # topics in a bag
pyroboreplay analyze mission.bag --detect-gaps --detail  # findings to stdout

The Python package exports Mission, Event, Failure, Hypothesis, RootCauseAnalysis, Action, FleetStatistics, GeoHotspot (see src/pyroboreplay/__init__.py). The multispectral, retrospective-detection, world-knowledge, and prediction features live in the Rust core and are not yet exposed through these bindings.

Documentation

• docs/CLAUDE.md — product vision and architecture
• docs/API.md — Python and Rust API reference
• docs/ARCHITECTURE.md — phase-by-phase architecture
• docs/QUICKSTART.md — getting started
• docs/STATS_DASHBOARD.md — live metrics dashboard (replay --stats-dashboard)
• examples/ — runnable demos (cargo run --example root_cause_analysis_demo)
• ROADMAP_HONEST.md — implemented vs planned, technical debt
• CHANGELOG.md — notable changes

Contributing

Contributions welcome — see CONTRIBUTING.md for setup, conventions, and PR guidelines. Bugs and ideas: GitHub Issues; questions: GitHub Discussions.

Part of the robotics & autonomy family

PyRoboReplay is one of six complementary, composable tools in the Mullassery robotics & autonomy family. Together they form a single stack: perceive → map → simulate → learn → debug → coordinate the fleet.

Each works standalone, but they're complementary by design. Browse all projects →

License

Apache License 2.0 — see LICENSE.