shortsmaker 0.2.0

Turn a long interview into publish-ready vertical clips: cut on sentence boundaries, reframed to 9:16, karaoke subtitles burned in.

🎬 shortsmaker

A long interview goes in. Publish-ready vertical clips come out.

Cut on sentence boundaries · reframed to 9:16 by body tracking · karaoke subtitles burned in

uvx shortsmaker run interview.mp4

A 15-minute interview becomes four or five postable clips in about five minutes, on a laptop with no GPU, for roughly $0.10 of API credit.

▶ Watch an example clip — 21 seconds, straight out of shortsmaker run, not retouched.

---

What it is for

The source it is built for is a filmed talking-head interview: a podcast, a long-form YouTube interview, a panel. One or two people, a static camera, a multi-camera edit.

Within that, it does one thing well: it finds the passages worth clipping, cuts them so they open and close on whole sentences, keeps the speaker in frame at 9:16, and burns in captions.

It is not a general-purpose video editor. On a vlog, a product demo or a sketch, the picture carries meaning that this pipeline deliberately ignores — it sends the model the audio, never the video. Expect poor results, and see SPEC.md for why that trade was made.

---

Install

Requires Python 3.12+, uv, and FFmpeg built with libass (apt install ffmpeg, brew install ffmpeg — the distro build is fine).

uvx shortsmaker run interview.mp4     # run it without installing anything
uv tool install shortsmaker           # or install the command for good

The font and the pose model are vendored inside the package: nothing is downloaded on first run.

Configure

Two keys, read from environment variables:

export OPENAI_API_KEY=sk-...     # Whisper — the transcript
export GEMINI_API_KEY=AIza...    # Gemini — choosing the passages

Get them from platform.openai.com and aistudio.google.com.

The tool never reads a .env file, and that is deliberate: a CLI runs in whatever directory you happen to be standing in, so a tool that loads .env from the working directory would silently pick up a file you did not know was there. Keys come from the environment, and how they get there is your business — export, a CI secret, a container's --env-file, or direnv loading a .envrc when you cd into the project. For local work, direnv is the comfortable answer; copy .envrc.example and run direnv allow.

What it costs

Only two of the five stages call a paid API, and both cache their result — so re-rendering a dozen times while you tune the look costs nothing.

Source length	Whisper	Gemini	Total
15 minutes	~$0.09	a few cents	~$0.10
1 h 40	~$0.61	a few cents	~$0.65

Whisper is billed on audio duration ($0.006/min). Gemini is negligible beside it, because it is sent the audio and the transcript — never the video.

No key, no spend: --mock fakes both APIs with realistic, deterministic data, and the full pipeline runs offline. It is how the test suite runs, and the fastest way to see what the tool actually does.

shortsmaker run interview.mp4 --mock

---

Use

shortsmaker run interview.mp4              # everything → clips/
shortsmaker run interview.mp4 -n 3         # keep only the 3 best-scoring passages
shortsmaker run interview.mp4 -f 1:1 -l en # square, English

What you get

clips/
  01-s0.94-le-vrai-luxe.mp4                 ← index, score, and the title the model wrote
  02-s0.88-pourquoi-il-a-dit-non.mp4
  03-s0.71-UNANCHORED-la-premiere-usine.mp4
.shortsmaker/                               ← intermediates, cached, safe to delete
  transcript.json   analysis.json   segments.json   reframe/

The filenames make a directory listing a ranking: index, score, title. A clip flagged UNANCHORED is one whose quote could not be located in the transcript, so its bounds fell back to the model's rough guess — expect a ragged cut. It is kept and flagged, never dropped silently, because the decision to throw a clip away is yours.

Options

	Default
-f --format	9:16	9:16 TikTok/Reels/Shorts · 4:5 Instagram · 1:1 LinkedIn/X · 16:9 keep the source shape (no crop, tracking skipped)
-l --language	fr	The spoken language, and the language of the generated titles. Also en, es, de, it, pt, nl.
-n --clips	all	Keep only the N best-scoring passages.
--min-score	none	Keep only passages scoring at least this. Combines with -n.
-o --out	clips/	Where the finished clips land.
-w --work-dir	.shortsmaker/	Where intermediates are cached.
-j --jobs	auto	Clips rendered in parallel.
--mock	off	Fake the APIs. No key, no cost.
--force	off	Ignore the cache and re-run the stage — including the paid ones.
-v --verbose	off	Show the underlying ffmpeg commands and the raw API traffic.

shortsmaker --help works on every command, and every default carries its reasoning.

One stage at a time

The pipeline is five stages, and they are separate commands for one reason: only the first two cost money. They cache, so everything downstream can be re-run for free.

shortsmaker transcribe interview.mp4   # Whisper   → transcript.json   ($, cached)
shortsmaker analyze    interview.mp4   # Gemini    → analysis.json     ($, cached)
shortsmaker show       interview.mp4   # print what was found, with scores — nothing rendered
shortsmaker reframe    interview.mp4   # MediaPipe → the crop plan
shortsmaker render     interview.mp4   # FFmpeg    → clips/

Re-selecting and restyling never call an API:

shortsmaker analyze interview.mp4 --min-score 0.8              # change your mind — free
shortsmaker render  interview.mp4 --size 96 --accent "#00E5FF" # restyle — free

analyze writes everything it found to analysis.json and never narrows it; -n and --min-score select from that. A stricter filter is a re-read, not a re-analysis.

As a library

Importing the package does not turn it into a CLI: nothing is written to stdout, and the stages log through the standard logging module, silent until your application adds a handler.

from shortsmaker import Paths, Settings
from shortsmaker.stages import analyze, transcribe

transcript = transcribe.transcribe(paths, Settings())
segments = analyze.analyze(paths, transcript, Settings(), clips=3)

See the package docstring for the full loop.

---

How it works

Gemini decides what to cut. Whisper decides when. Neither is asked to do the other's job.

Gemini reliably picks the right moments but cannot place them in time — its timestamps drift by around two seconds, enough to open a clip mid-sentence. So it is never asked for a timestamp. It quotes the words, and the quote is looked up in Whisper's word-level transcript. Cut points land on real word boundaries by construction rather than by luck.

The reframe tracks bodies, not faces — a face vanishes in profile, a body does not — decides per shot rather than per frame, and never guesses who is speaking: two people in frame means both are shown.

→ SPEC.md has the flowchart and the reasoning, stage by stage. Four spikes were built and measured before any of this was written, and nearly every rule in the code replaced one that made the clips worse.

Performance

Measured on a 16-core CPU with no GPU: reframing runs at 0.71× realtime, rendering at 0.45×, so local processing costs roughly 1.2× the source duration. Transcription and analysis are network-bound — about 90 seconds for a 15-minute video.

---

Development

uv sync                                     # runtime deps + pytest
uv run pytest                               # tests

uvx ruff@0.15.21 check .                    # lint
uvx ty@0.0.58 check                         # types
uvx bandit@1.9.4 -c pyproject.toml -r src   # security

These are, verbatim, the four commands CI runs — green here means green there. The conventions they enforce, and the traps worth knowing before changing anything, are in CLAUDE.md.

What changed between versions: CHANGELOG.md. To cut a release: PUBLISHING.md — the short version is push a tag.

Licence and credits

The code is MIT — see LICENSE.

The example clip is not. It is a short excerpt of an interview with Bernard Arnault from Legend (Guillaume Pley), reproduced solely to illustrate what the tool outputs. All rights remain with its owners; it is not covered by this project's licence and is not redistributable as part of it.

Vendored assets carry their own licences: Lato (Łukasz Dziedzic, SIL Open Font License 1.1) and MediaPipe's pose_landmarker (Google, Apache 2.0).