paces

Turn instructional media into structured, interactive learning material. Put it through its paces.

Take a video of someone teaching something — a dance routine, a kata, a recipe — plus, optionally, notes and a steering prompt. paces segments it into named steps, builds a structured step document (an AST for step-by-step instruction), and renders that into learning material: a practice page with counts and deep links today, other guides later.

pip install paces

Quick example

from paces import segment, to_document, render_html

seg = segment(
    "https://youtu.be/q_TUyxUhoEw",
    steps=[
        ("Mise en place", 2),
        ("Pas pieds pointe et ronde", 6),
        ("Soleil avec les bras", 4),
        ("Déhanchés", 8),
    ],
    grid={"unit": "eight", "subdivisions": 8, "tempoBpm": "129.2", "origin": "51.2"},
)
doc = to_document(
    seg,
    doc_id="que-calor",
    title="Chorégraphie Que Calor",
    source="https://youtu.be/q_TUyxUhoEw",
)
open("page.html", "w").write(render_html(doc))

The page lists every step with its counts, links each one back into the video (both the at-tempo run-through and the slow breakdown, when both are known), and — because the document carries a metric grid — includes a count-along transport that paces you through the routine at the measured tempo.

Same thing from the shell:

paces segment VIDEO_URL --steps steps.json --grid grid.json --output seg.json
paces to-document seg.json --source VIDEO_URL --title "My routine" --output document.json
paces suggest-excerpts document.json --output document.json   # mark each block's loop window
paces derive document.json --media routine.mp4   # real loop clips + gifs + posters
paces render document.json --output page.html

derive (pip install paces[media]) cuts a loopable mp4, a palette-quality gif and a poster for every excerpt-bearing span, writes them to media/ next to the document, and the practice page embeds them as looping clips. Crop recipes persist in a hand-overridable document.recipes.json sidecar; the subject_locator= seam (default: no crop) is where pose-based auto-crop plugs in. Design record: docs/adr/0005-media-derivation.md.

Auto-crop to the people in frame with pip install paces[pose], then paces derive doc.json --media routine.mp4 --subject-locator paces.pose:rtmlib_pose (or subject_locator=paces.pose.rtmlib_pose from Python). It probes each excerpt window at ~5 fps and reports every person it sees; the crop policy stays in the core, so two people in frame get one box around both. The extra itself is rtmlib (Apache-2.0, pure Python) and onnxruntime (MIT), with model weights downloaded on first use. Detection is YOLOX (Apache-2.0); what is barred from every extra here — rather than quarantined into one — is the ultralytics distribution, which is AGPL-3.0.

What it pulls in is a different question, and worth stating plainly: rtmlib requires opencv, and opencv's bundled FFmpeg is GPL-3.0-or-later on macOS wheels of the versions measured (4.12.0.88 / 4.13.0.92) (built --enable-gpl with libx264/libx265) though LGPL-2.1-or-later on manylinux and Windows — a per-version fact, not a per-platform one: the 5.0.0.93 macOS x86_64 wheel ships no FFmpeg at all. This is measured from the shipped binaries — the wheels' own LICENSE-3RD-PARTY.txt never mentions x264. paces[media] already brings such a wheel, so [pose] adds a second copy rather than a higher tier. Worse: rtmlib's own metadata requires both opencv-python and opencv-contrib-python, unpinned, so a plain pip install paces[media,pose] ends up with two distributions owning one cv2 — harmless until either is uninstalled, at which point the survivor's cv2 can be left with files missing (issue #20). rtmlib only calls plain cv2 APIs (VideoCapture, dnn.readNetFromONNX, drawing helpers — nothing contrib-only), so [media]'s opencv-contrib-python already covers it; to keep the closure single-provider, install in two steps instead of one:

pip install "paces[media]"                    # opencv-contrib-python, the fleet's one cv2
pip install --no-deps "rtmlib>=0.0.16,<0.1"   # skip rtmlib's own opencv-* re-declaration
pip install onnxruntime tqdm                  # rtmlib's other real deps (numpy already arrives via opencv)

paces.pose.check_pose_requirements() reports what you have, and names the repair command if both providers are already present.

How it thinks

Analysis and rendering are separate phases with a serialisable document between them — like a parser emitting an AST and a backend interpreting it. Renderers depend on the document, never on the analyser.

Segmentation is a seam, not a stage. segment(media, segmenter=...) — segmenters are registered capabilities, the default follows from what is present, and "the user typed the boundaries" is a first-class segmenter, not a fallback. A segmenter that cannot name steps returns honest unnamed boundaries (flags: ['naming-abstained']) rather than inventing names.

The document keeps what the learner actually counts. A dance step lasts "4 eights", not "14.86 seconds" — seconds are derived from the metric grid (tempo + origin), never stored. A step can have several source spans (the run-through and the breakdown are the same step seen twice). Uncertainty is content (OpenQuestion), and human edits are protected from regeneration (Lock).

The pieces

Status

Young and moving. The document schema is validated by round-tripping a real proof of concept (an interactive dance-practice page) through it — see tests/test_roundtrip_poc.py. Media derivation (auto-cropped looping clips), intrinsic segmenters (scene/beat/speech detection), and the evidence layer are designed (see docs/) and arrive next.