annie 1.8.1

Local-first browser UI to explore, inspect, and validate a video annotation dataset

Annie visualizes a video dataset alongside its frame-wise annotations and lets you curate and (soon) author them — all in one local, browser-based UI that runs on a single machine. The motivating datasets are CMU-MOSEI and First Impression V2 (face detections + derived face tracks), but Annie is dataset-agnostic.

Screenshots

Home tab — landing page with a summary card for each tab.

Browse tab — per-video rows with original clip, annotated frame strip, and label tags, above collapsible Manipulate / Filter / View panels. Click a row's top-right corner to queue it for the Annotator.

Annotator tab — where supervision is entered, one task at a time. The task switch offers only the tasks your sources support: Curation (shown), Protagonist review, and Segment review.

Features

See the playbooks for screen-by-screen walkthroughs of the main flows.

Highlights

• Extensible data sources — a dataset is an ordered set of sources; a CSV joins to videos by a chosen key column, exposing its value columns as Browse tags and filter facets (e.g. Sentiment: negative, Angry: 0.33). Dataset-agnostic by construction.
• Stem matching — videos pair with vdet/track files by filename stem (exact + prefix, longest-stem-first), aggregated into one row per video.
• Composable filtering — filter by vdet/track coverage, review verdict, notes, annotator selection, and label values; facets combine with AND, label values OR.
• Frame-accurate decode — torchcodec with exact mode for the annotator and approximate for fast scrubbing.
• Render-on-demand — a background job burns annotations into a browser-playable clip (libx264 via FFmpeg, audio muxed back); temp clips auto-purge on a TTL.
• Protagonist correction — pick the true protagonist; the fix is written to a separate _manual file (resolution manual ▸ source ▸ -1) so human judgement never overwrites the pristine heuristic record, and the resolved datasource exports to a standalone CSV.
• Segment review — for a long video cut into per-clip segments, accept or drop each clip while comparing competing start/end bands (e.g. a cut prediction vs a GT reference) side by side; decisions persist for resumable passes and export as separate accepted / dropped CSVs.
• Event annotation — a video-editor-style timeline: watch one video, scrub it frame-by-frame (±5 s, frame step, I/O in-out marks), and draw labelled events on stacked participant lanes (categories like Mother / Baby, defined once per dataset and picked — never re-typed — so labelling stays consistent; press 1–9 to switch the active participant). Each event carries a name, note, colour, and arbitrary key/value attributes; everything saves to the session DB as you go and exports to nested JSON (for code) and flat CSV (for a spreadsheet), per video or per session. Zoom the timeline and pan with the scrollbar or trackpad; during playback the window follows the red playhead (page-flip), and panning by hand pauses following until you re-enable it.

Installation

Windows (for non-technical users)

If you just want to use Annie on Windows — no terminal, no Python, no FFmpeg — use the one-click launcher in launcher/windows/: install Docker Desktop once, double-click Annie, and pick your video folder when asked. It opens in your browser and updates itself on each launch. See launcher/windows/README-Windows.txt for the three-step guide. The developer install paths below are not needed for this.

Prerequisites

Required for the source and PyPI install paths. Docker bundles everything automatically.

From PyPI

uv pip install "annie[all]"     # core + torch / torchcodec for frame decode & render
annie                           # starts the UI at http://127.0.0.1:8080

From source (development)

git clone https://github.com/fodorad/Annie
cd Annie
make dev      # installs annie[all,dev] in editable mode (torch + torchcodec included)
make run      # starts the UI at http://127.0.0.1:8080

Docker

No Python, uv, or FFmpeg required on the host — Docker bundles everything.

git clone https://github.com/fodorad/Annie
cd Annie
cp .env.example .env   # fill in your HDD paths
make docker-build      # ~5–10 min first time (downloads CPU torch)
make docker-run        # starts the UI at http://localhost:8080

Or pull the pre-built image without cloning:

curl -O https://raw.githubusercontent.com/fodorad/Annie/main/docker-compose.yml
curl -O https://raw.githubusercontent.com/fodorad/Annie/main/.env.example
cp .env.example .env   # fill in your HDD paths
docker compose up      # pulls fodorad/annie:latest automatically

Annie state (logs, session DBs, render cache, saved configs) is stored in ./annie-home/ next to docker-compose.yml by default, so all files are directly accessible from Finder / Explorer. Exported review CSVs land in annie-home/tmp/. Override the path with ANNIE_HOME_HOST in .env.

macOS + external HDDs: If /Volumes is already in Docker Desktop → Settings → Resources → File Sharing, individual drives under /Volumes work without any additional configuration.

Extras

Architecture

Annie is a single process with a strict layered architecture; each layer calls only the layer directly beneath it, enforced by import direction:

UI layer          annie/app.py, annie/pages/*          (NiceGUI tabs)
   │  calls down only
Service layer     annie/dataset/*, annie/media/*        (scanning, rendering, filtering, …)
   │
Domain layer      annie/core/models.py, annie/parsers/* (pure data, no I/O frameworks)
   │
Infrastructure    annie/core/config.py, annie/core/theme.py,
                  annie/dataset/storage.py (SQLite), annie/media/decode.py (torchcodec)

The UI never imports sqlite3 or torchcodec directly — it calls a service function. The core layers install without a heavyweight torch dependency; only the frame decode and render pipeline need the media extra.

Annotation formats

Both .vdet and .track files share one 17-column CSV schema (CRLF, with header):

frame_id, source, score, x, y, w, h,
left_eye_x, left_eye_y, right_eye_x, right_eye_y,
nose_x, nose_y, left_mouth_x, left_mouth_y, right_mouth_x, right_mouth_y

• .vdet — all raw detections for a video; may have several rows per frame.
• {video_id}__track{N}.csv — one tracked face across frames (one row/frame).
• protagonist CSV — e.g. protagonist_track_heuristic.csv with uuid,track_id, the active protagonist per video (-1 = none). The key and track-id columns are chosen when the source is added; manual corrections go to the _manual sibling.
• label CSV — any CSV; pick a key column (joined to the video id) and value columns to surface as Browse tags and filter facets.
• segmentation CSV — e.g. review_band.csv; one row per clip keyed by video_id + segment_id, with one or more named start/end bands (in seconds) and an optional video_path column. Drives the Annotator's Segment-review task.

Development

git clone https://github.com/fodorad/Annie
cd Annie
make dev      # install in editable mode with all extras
make check    # lint + type-check + test + docs (mirrors CI)
make run      # start the app at http://127.0.0.1:8080

Every screenshot in this README and in the docs is generated from the running UI, so it never drifts from what Annie actually looks like. After changing a screen, refresh them with make screenshots (needs make install-docs once) and commit the updated PNGs.

Tip — skip re-clicking paths on every restart: set ANNIE_* environment variables and Annie will seed those sources automatically at launch. This is equivalent to manually adding sources on the Dataset tab, but saves time during development when you always work with the same dataset.

export ANNIE_VIDEO_DIR=/path/to/video
export ANNIE_VDET_DIR=/path/to/vdet
export ANNIE_TRACK_DIR=/path/to/track
export ANNIE_PROTAGONIST_CSV=/path/to/protagonist_track_heuristic.csv
export ANNIE_LABEL_CSV=/path/to/label.csv
make run

Sources seeded via env vars are session-only; curation and corrections always persist. Save your source configuration from the Dataset tab to a named config file to reload it in one click without relying on env vars. Every setting has a corresponding ANNIE_* variable (see annie/core/config.py).

See CONTRIBUTING.md for the commit convention and release flow.

Documentation

• API Reference

Related Projects

• exordium — multimodal feature extraction (the detectors/trackers that produce Annie's .vdet/.track inputs).

Contact

Ádám Fodor — adamfodor.com · fodorad201@gmail.com

License

MIT