shortsmaker 0.1.1

Turn a long interview into publish-ready vertical clips: cut on sentence boundaries, reframed to 9:16, karaoke subtitles burned in.

🎬 shortsmaker

A long interview goes in. Publish-ready vertical clips come out.

Cut on sentence boundaries · reframed to 9:16 by body tracking · karaoke subtitles burned in

uvx shortsmaker run interview.mp4

A 15-minute extract of a podcast becomes four or five clips you could post, in about five minutes, on a laptop with no GPU. French by default (-l fr); also English, Spanish, German, Italian, Portuguese and Dutch.

▶ Watch an example clip — 21 seconds, straight out of shortsmaker run, not retouched. It shows the three things the pipeline exists to get right: the clip opens and closes on a whole sentence, the crop holds on the speaker and jumps only where the show's own editor cut, and the caption highlights each word as it is spoken.

---

Install

Needs Python 3.12+, uv, and FFmpeg built with libass (the distro package is fine — apt install ffmpeg).

uvx shortsmaker run interview.mp4     # run it without installing anything
uv tool install shortsmaker           # or install the command for good

The font and the pose model are vendored inside the package: nothing else to download.

API keys

Key	For
OPENAI_API_KEY	Whisper transcription
GEMINI_API_KEY	Choosing the segments

These are the only two stages that cost money, and they cache. Everything else runs locally and free. No keys to hand? --mock fakes both APIs with realistic data, so the whole pipeline runs offline and free — useful for trying it out, and for CI.

shortsmaker run interview.mp4 --mock

---

Use

shortsmaker run interview.mp4              # everything → clips/
shortsmaker run interview.mp4 -n 3         # keep only the 3 best passages
shortsmaker run interview.mp4 -f 1:1 -l en # square, English

	Default
-f --format	9:16	9:16 TikTok/Reels/Shorts · 4:5 Instagram · 1:1 LinkedIn/X · 16:9 keep the original shape (no crop, tracking skipped)
-l --language	fr	The language spoken — and the one titles and summaries are written in.
-n --clips	all	Keep only the N best-scoring passages.
--min-score	none	Keep only passages scoring at least this. Combines with -n.
-j --jobs	auto	Clips processed at once.
-o --out	clips/	Where the finished clips land.
-v --verbose	off	Show the ffmpeg commands and the raw API traffic.
--mock	off	Fake the APIs. No key, no cost.

--force has no short flag on purpose: it re-runs the paid stages, so it should cost a moment's thought.

Or one stage at a time. Only the first two cost money, so you can re-run the last ones as often as you like while tuning the look:

shortsmaker transcribe interview.mp4        # Whisper   → transcript.json   (the clock)
shortsmaker analyze    interview.mp4        # Gemini    → analysis.json     (paid, kept whole)
shortsmaker show       interview.mp4        # print what was found, with scores
shortsmaker reframe    interview.mp4        # MediaPipe → the crop plan
shortsmaker render     interview.mp4        # ffmpeg    → clips/

shortsmaker render interview.mp4 --size 96 --accent "#00E5FF"   # restyle, free
shortsmaker analyze interview.mp4 --min-score 0.8               # change your mind, free

Choosing which passages to keep costs nothing. analyze writes everything it found and never narrows it; -n and --min-score select from that, without a second API call.

shortsmaker --help on anything — the defaults carry their reasoning.

As a library

Importing it does not turn it into a CLI: nothing writes to stdout, and the stages log through the standard logging module, silent until your application adds a handler.

from shortsmaker import Paths, Settings
from shortsmaker.stages import analyze, transcribe

transcript = transcribe.transcribe(paths, Settings())
segments = analyze.analyze(paths, transcript, Settings(), clips=3)

See the package docstring for the full loop.

---

How it works, and why

→ SPEC.md — the flowchart, and the reasoning stage by stage. Four spikes were built and measured before any of this was written, and nearly every rule in the code replaced one that made the clips worse. The three decisions worth knowing:

Gemini decides what to cut. Whisper decides when. Gemini picks the right moments but cannot place them in time — its timestamps drift by ~2 s, and one run answered in MM.SS while the prompt demanded seconds, producing five 1-second clips that passed every check in silence. So it is never asked for timestamps: it quotes the words, and the quote is looked up in Whisper's word-level transcript. Bounds land on real word boundaries by construction.

Gemini gets the audio, never the video. The source is a talking head. The frames carry nothing the audio doesn't, but they dominate the token bill. What decides whether a Short lands — a laugh, the pause before a punchline — is in the audio, and Gemini hears it natively.

The reframe tracks bodies, and never guesses who is speaking. Face detection missed the guest on a wide shot (in profile, head down) and hallucinated a box on the background. Bodies don't vanish in profile. And when two people are in frame, we show both rather than pick one.

Performance

Measured on a 16-core CPU, no GPU: reframe 0.71× realtime, render 0.45×, so local processing runs at about 1.2× realtime. Transcription and analysis are network-bound — roughly 90 seconds for a 15-minute video.

---

Development

uv sync                                     # runtime deps + pytest
uv run pytest                               # tests

uvx ruff@0.15.21 check .                    # lint
uvx ty@0.0.58 check                         # types
uvx bandit@1.9.4 -c pyproject.toml -r src   # security

These are, verbatim, the four commands CI runs — see ci.yml. Green here means green there. The conventions they enforce, and the traps worth knowing before changing anything, are in CLAUDE.md.

To cut a release, see PUBLISHING.md. The short version: push a tag.

---

Licence and credits

The code is MIT — see LICENSE.

The example clip in docs/ is not. It is a short excerpt of an interview with Bernard Arnault from the show Legend (Guillaume Pley), reproduced solely to illustrate what the tool outputs. All rights remain with its owners; it is not covered by this project's licence and is not redistributable as part of it.

The vendored assets carry their own licences: Lato (Łukasz Dziedzic, SIL Open Font License 1.1) and MediaPipe's pose_landmarker (Google, Apache 2.0).