Skip to main content

fsd-disengagement-studio

PyPI CI

Catalog and classify FSD disengagements from your saved Tesla Dashcam clips. Newly viable in May 2026 because firmware 2025.44.25+ embeds frame-perfect SEI metadata (including AutopilotState) in saved clips, and Tesla published the proto schema in teslamotors/dashcam (Dec 4, 2025).

Install

# From PyPI (once published)
pip install fsd-disengagement-studio              # core CLI + dashboard
pip install 'fsd-disengagement-studio[sei]'       # +protobuf/pandas to parse Dashcam SEI
pip install 'fsd-disengagement-studio[dev]'       # +pytest, ruff, httpx

Then try the dashboard with no Tesla and no API keys:

fsd-studio demo                                   # seeds + serves at http://127.0.0.1:8000/

The wave-3 spec at ../ideas/v2/wave3/fsd_disengagement_studio.md walks through the full design rationale.

What it does

For each saved Dashcam event:

  1. Parse the SEI metadata stream to find frames where AutopilotState transitions from SELF_DRIVING/AUTOSTEER to NONE while vehicle_speed_mps > 1. (The speed gate filters out parking-lot manual-takeover events.)
  2. For each disengagement, sample keyframes from front + back cams in a ±5s window around the transition.
  3. Classify the cause via VLM (gpt-4o-mini default, ~$0.0008/event) into a structured DisengagementCause enum (phantom_brake, lane_confusion, occluded_sign, hov_decision, driver_pre_empt, traffic_light_hesitation, construction_zone, sun_glare, weather, unclear, other).
  4. Persist to SQLite with firmware version + GPS + clip path + confidence.
  5. Serve a FastAPI + HTMX dashboard showing per-firmware regression: which cause categories are improving, which are worsening across firmware versions.

Composability with the v3 stack

This project is the second consumer of tesla-clip-tools (the first being sentrytriage). The library provides:

  • tesla_clip_tools.sources.tesla.TeslaSource — walks the canonical TeslaCam folder layout
  • tesla_clip_tools.sampler.sample_keyframes — frame extraction via imageio (bundled ffmpeg)
  • tesla_clip_tools.vlm.openai.OpenAIBackend — generic VLM call parameterised on response_format
  • tesla_clip_tools.sei.iter_sei_payloads / sei_dataframe — the SEI byte-walker

This repo adds: the FSD-specific schema, the transition detector, the per-firmware regression dashboard, and the disengagement-cause classifier prompt.

Status — v0.4 alpha

What works today:

  • Path B auto-saves the dashcam clip (added v0.4): fsd-studio listen --auto-save POSTs to Tesla's manual_save_dashcam_event Fleet API the moment a disengagement is queued, so Path C is guaranteed to find the matching .mp4 in SavedClips/ later. Per-vehicle 30s cooldown stops a re-broadcast burst from spamming the API; Fleet API errors are logged but never crash the listen loop. --mock-fleet runs a realistic dry-run without contacting Tesla. The dashboard /pending page surfaces the running auto-save counter ("Auto-saves issued: N").
  • All three paths collaborate (v0.3): Path A (manual save), Path B (real-time trigger), and Path C (bulk backfill) compose as one system.
    • fsd-studio listen queues real-time disengagements to a JSONL log
    • fsd-studio backfill --pending-log <path> reads the log, matches each newly-classified DisengagementEvent to a queued record by ±300s (configurable) timestamp window, marks the record with the matched event's ID, and atomically rewrites the log
    • Dashboard GET /pending?unmatched_only=true (default) shows queued records still waiting for a clip; GET /pending/stale?stale_after_hours=24 surfaces missing-clip warnings (Path B fired but no clip was ever saved)
  • Path B real-time trigger (v0.2): fsd-studio listen --ha-url ... --ha-token ... subscribes to HA's tesla_fleet_telemetry event bus, runs a stateful DisengagementDetector with cooldown + low-speed gate, appends each detected transition to a JSONL log. Detector handles both explicit autopilot_state transitions and inferred-from-self_driving_miles plateaus. Fully testable with MockTelemetryStream.
  • Pure-Python find_disengagements() finds engaged → disengaged transitions in any sequence of FrameTelemetry tuples (testable without a Tesla via mocked sequences)
  • DisengagementClassification Pydantic schema with 11 cause categories
  • DisengagementEvent SQLModel + Store with upsert, list, and per-firmware aggregation
  • fsd-studio CLI with classify, backfill, serve commands
  • fsd-studio serve starts a FastAPI + HTMX + ECharts dashboard at localhost:8000
  • Test suite covers the schema, transition detector, store aggregation, and FastAPI routes (TestClient — no real server needed)

What needs a real Tesla + the [sei] extra:

  • iter_disengagements_from_clip() — calls sei_dataframe(mp4_path) which requires the [sei] extra (pip install 'fsd-disengagement-studio[sei]') plus a one-time protoc --python_out=. dashcam.proto to generate dashcam_pb2.py in tesla-clip-tools.
  • The backfill CLI command — needs real saved Dashcam clips with SEI to do meaningful work.

What's coming in v0.5+ (per wave-3 spec):

  • Per-route map (Leaflet) showing where disengagements cluster
  • Per-cause Sankey across firmware versions
  • Regression alerts (push when a cause spikes vs trailing N-week baseline)
  • Reviewer-feedback fine-tuning (thumbs-up/down on each event tunes a small classifier on top of the VLM embeddings)

Path B auto-save (v0.4) -- usage

export TESLA_FLEET_ACCESS_TOKEN=...   # from your Fleet API OAuth flow
export TESLA_VEHICLE_ID=...           # the vehicle id (or VIN) to target
export HA_URL=http://homeassistant.local:8123
export HA_LONG_LIVED_TOKEN=...

# Auto-save is on by default; the matching clip lands in SavedClips/.
uv run fsd-studio listen

# Dry-run (no real Fleet API call):
uv run fsd-studio listen --mock-fleet --vehicle-id rehearsal

# Disable entirely (v0.3 behavior):
uv run fsd-studio listen --no-auto-save

The dashboard's /pending page renders an "Auto-saves issued: N" stat panel fed by a sidecar <pending-log>.autosave-stats.json that the listener writes after each event. JSON consumers can hit GET /api/autosave-stats.

Quick start (without a Tesla)

git clone https://github.com/Raymondriter/fsd-disengagement-studio.git
cd fsd-disengagement-studio
uv sync --extra dev
uv run pytest                                  # 15+ tests, no fixtures needed
uv run fsd-studio serve --port 8000           # empty dashboard at http://localhost:8000

Quick start (with a Tesla)

uv sync --extra sei --extra dev
# Generate the SEI proto module (one-time):
cd ../tesla-clip-tools/src/tesla_clip_tools
protoc --python_out=. dashcam.proto
cd -

export OPENAI_API_KEY=sk-...
uv run fsd-studio backfill /path/to/SavedClips --firmware 2026.14.3
uv run fsd-studio serve

Important caveats

  • SEI is only in Dashcam clips (SavedClips, RecentClips), not Sentry events. The wave-3 SEI deep-dive confirmed this. So this tool reads from SavedClips/ (and RecentClips/), NOT SentryClips/. The SavedClips/ folder layout is identical to SentryClips/ — tesla_clip_tools.sources.tesla.TeslaSource handles either.
  • Path B (Fleet Telemetry triggered save) is fragile in v0.1. Tesla hasn't shipped a first-class autopilot-state transition signal; it has to be inferred from SelfDrivingMilesSinceReset rate-drop. v0.1 ships only Path A (manually-saved clips) + Path C (bulk historical backfill). Path B is v0.2 opt-in.
  • Legal risk for individual-owner use is low. Owner owns vehicle-generated data per Tesla's Privacy Notice + Fleet API Terms; teslafsdtracker.com and MATT3R K3Y have published cross-owner FSD-disengagement data since 2020 with no Tesla legal action; Tesla itself publishes aggregate data in its FSD Vehicle Safety Report. Single-VIN by default. Footer disclaimer in the dashboard reminds users not to generalize from n=1 in public posts.

Screenshots

The dashboard rendered against synthetic demo data (fsd-studio demo):

Dashboard with per-firmware regression bars and recent events table

Filterable events table — narrow by firmware version and disengagement cause

Path B pending queue — disengagements waiting on a saved-clip match

Changelog

See CHANGELOG.md. Versions follow Keep a Changelog and the project uses SemVer.

License

MIT. See LICENSE.

CI

ci

GitHub Actions runs ruff + pytest on Python 3.12 and 3.13 against every push and PR. See .github/workflows/ci.yml. Until tesla-clip-tools is published to PyPI, the standalone CI strips the [tool.uv.sources] table and resolves it as a regular dependency; the workspace-level monorepo CI at C:\Dev\tesla\.github\workflows\ci.yml keeps using the path-editable sibling.

Metadata

Release files for fsd-disengagement-studio 0.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fsd-disengagement-studio 0.5.0
File Size Uploaded Provenance Actions
fsd_disengagement_studio-0.5.0.tar.gz 54.2 kB Without provenance Details for fsd_disengagement_studio-0.5.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for fsd-disengagement-studio 0.5.0
File Interpreter ABI Platform Actions
fsd_disengagement_studio-0.5.0-py3-none-any.whl Python 3 none any Details for fsd_disengagement_studio-0.5.0-py3-none-any.whl

Total release size: 105.0 kB

Release files / fsd_disengagement_studio-0.5.0.tar.gz

Download URL fsd_disengagement_studio-0.5.0.tar.gz
Size 54.2 kB
Tags Source
SHA-256 checksum
How to use checksums
80fe984b323a1f090cad58b295cb2d982924979b0a9b5aaa88fc500d0016c8f7
BLAKE2b-256 checksum
How to use checksums
30bdd25ab919126dfd012a62151f52a6b5d6dc6cbafcd0573904e2f4e62f5b38
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.12 {"installer":{"name":"uv","version":"0.11.12","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / fsd_disengagement_studio-0.5.0-py3-none-any.whl

Download URL fsd_disengagement_studio-0.5.0-py3-none-any.whl
Size 50.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a705599d8ca90ad7216150abcdcd96f605697b54a1352e0c579fef67aae95b1a
BLAKE2b-256 checksum
How to use checksums
c0122046876c4c119f3962a4085cdbfaf076617bd9b4f1ac04f03917af8ad18f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.12 {"installer":{"name":"uv","version":"0.11.12","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for fsd-disengagement-studio 0.5.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.5.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page