Guetteur (Team Drift): active perception with receipts

Guetteur is French for "lookout". An agent watches a space through depth only, decides what to look at next using OpenCV 5 evidence, stops at a human gate before anything outward-facing happens, and attaches a re-runnable receipt to every claim it makes.

Status (2026-10-15): submitted to the OpenCV AI Competition powered by AWS. Devpost entry · Narrated demo (3:52) · Submission receipt. AWS judge demo deployed; physical depth capture verified. Try the live dashboard. The published accuracy results remain synthetic-only. Three physical OAK-D depth takes (empty baseline, walkthrough, item removal) each contain 300 integrity-checked 640x400 frames; calibrated replay and real-world accuracy measurement remain open. The AWS demo runs built-in synthetic scenes, not a live camera feed. See current evidence.

Built for the OpenCV AI Competition 2026 (Agentic Vision path). Rules notes with section citations: docs/RULES_NOTES.md.

What it does

depth frames ──> OpenCV 5 perception ──> evidence (JSON, no pixels) ──> planner ──> action ──> next perception call
                 inpaint, medianBlur,                                  rule-based    recheck_zone / pause_zone /
                 morphology, connected                                 (LLM slot)    compensate_drift / degrade_zone
                 components, Farneback                                               request_approval  (human gate)
                                                                                          │
                                           every claim ──> claim.json ──> receipts ──> prove-it-done verifier

• Ambiguous evidence changes the next tool call. A 25 mm change is ~3.5 sigma in one frame; the planner asks recheck_zone(n=7) and OpenCV 5 re-runs on the integrated window before anyone is bothered.
• Presence is not a change. A person-sized blob nearer than the scene pauses that zone.
• Sensor trouble is not a scene event. Noise bursts degrade the zone, gain drift is compensated, lost depth escalates as a camera fault.
• Default is deny. If no human answers the approval request, nothing is published.
• Claims carry receipts. Each claim is re-checked by two independent estimators (numpy-only, and OpenCV frame statistics) against the pinned input, and the vendored prove-it-done verifier judges the ledger. Claims doctored beyond the tolerance (max(8 mm, 25% of the value)) fail: inflated, pointed at the wrong pixels or time, or re-pointed at another recording. Smaller inflations pass by design.

Quick start (macOS/Linux, Python 3.12, uv)

uv sync                                    # pinned: opencv-python-headless==5.0.0.93, numpy==2.5.4, pytest==9.1.1 (see uv.lock)
uv run pytest                              # ~2-3 minutes; includes the real verifier and tamper tests
uv run guetteur scenarios                  # the 11 synthetic scenarios and their ground truth
uv run guetteur demo                       # narrated demo (the judge demo without a camera)
uv run guetteur run subtle_item_removed --verify     # one scenario: trace, alerts, score, receipts
uv run guetteur run person_takes_item --gate timeout # human gate times out -> nothing is published
uv run guetteur run subtle_item_removed --planner naive   # the threshold-and-alert baseline misses it
uv run guetteur eval                       # all scenarios x 3 seeds x 3 planners -> eval/results.json
uv run guetteur bench                      # per-frame runtime on this machine -> eval/runtime.json
uv run guetteur serve                      # the judge dashboard + API on http://127.0.0.1:8080/

From PyPI instead of a clone (Python 3.12): pip install guetteur, then the same commands without uv run (for example guetteur demo). Outputs go under the working directory.

Re-verify any ledger yourself: uv run guetteur verify-ledger runs/<run>/receipts.md (runs the vendored third_party/prove_it_done/verify_receipts.py, pinned to prove-it-done commit d897534).

Optional physical-camera capture

An opt-in, depth-only native recorder and operator run sheet are available in docs/REAL_DEPTH_CAPTURE.md. Hardware capture and clean shutdown are verified; calibrated replay remains open. This does not change the synthetic-only status of the accuracy evaluation.

Layout

Honest limits

• Accuracy metrics are synthetic-only. Physical depth recordings exist, but calibration, interval labels, and held-out replay are still required before a real-world accuracy claim.
• Depth cannot tell what an object is; claims say "a region became farther by 24 mm", not "a bottle was removed".
• Changes under ~1 sigma after filtering are invisible (documented miss: subthreshold_item_removed).
• The LLM planner is an interface plus tests with a fake model; no model is called.
• The edge/cloud approval path (DynamoDB, S3, authenticated routes) is code plus local tests only; the first AWS deployment is a public synthetic-demo endpoint (INFRA/README.md).
• The deployed AWS scope is API Gateway + Lambda arm64 running OpenCV and receipt checks. DynamoDB, S3, and authenticated real-camera approvals remain phase-2 code with local tests only.
• The dashboard gate selector simulates approve/deny/timeout; it is not a deployed authenticated human-approval workflow.
• The demo has API throttling but no reserved-concurrency cap because the account quota could not support the reservation.

License

MIT (see LICENSE). Vendored third_party/prove_it_done is MIT as well.