subreel

Turn speech audio into subtitle reels: green-screen H.264, transparent WebM/ProRes, with word-by-word karaoke highlight.

subreel is a small, dependency-free CLI that turns a voice-over into a subtitle clip you can drop straight onto a video timeline. It renders the subtitles onto a chroma-key green background (for editors that key out green) or onto a transparent canvas (for editors that take alpha directly), and it lights up each word as it is spoken — classic karaoke style.

Two ways to get the timeline:

1. subreel tts — the accurate path. Synthesize the speech with edge-tts and capture the per-word timestamps straight from the synthesis events. The subtitle timing is exact, because it comes from the same engine that produced the audio — zero ASR error.
2. subreel render --asr — the any-audio path. Run faster-whisper on any existing audio file to get word-level timestamps.

Features

• Three output flavours from one command:
  • Green-screen MP4 — H.264 / yuv420p, ready for chroma key in 剪映, Premiere, DaVinci, Final Cut.
  • Transparent WebM — VP9 with an alpha channel (yuva420p).
  • Transparent ProRes 4444 MOV — with alpha (yuva444p10le), for Premiere / Final Cut / DaVinci.
• Per-word karaoke highlight — each word is wrapped in an ASS \kf sweep, so the current and already-sung words are drawn in the highlight colour.
• edge-tts precise timelines — word timings captured from WordBoundary events, no speech recognition involved.
• Whisper ASR fallback — word-level recognition for audio you did not synthesize, with caching.
• Zero mandatory Python dependencies — ffmpeg is the only system requirement; edge-tts and faster-whisper are opt-in extras.
• A documented, portable timeline format — the words.json file.

Installation

1. ffmpeg (required)

subreel shells out to ffmpeg / ffprobe and checks for them at startup.

For CJK subtitles, install a CJK font as well (e.g. fonts-noto-cjk on Debian / Ubuntu); the default style uses Noto Sans CJK SC.

2. subreel

Packaging is CI-verified: pip install subreel works as soon as the first release lands on PyPI (see docs/releasing.md). Until then install straight from GitHub:

pip install "git+https://github.com/paxoscn/subreel.git"

With the optional features:

# edge-tts synthesis (subreel tts)
pip install "git+https://github.com/paxoscn/subreel.git#egg=subreel[tts]"

# faster-whisper ASR (subreel render --asr)
pip install "git+https://github.com/paxoscn/subreel.git#egg=subreel[asr]"

# both
pip install "git+https://github.com/paxoscn/subreel.git#egg=subreel[tts,asr]"

Quick start

A. Synthesize and render (most accurate)

# 1. make the voice-over plus its exact word timings
subreel tts \
  --text script.txt \
  --voice zh-CN-XiaoxiaoNeural \
  --rate +15% \
  --out voice.mp3
# -> voice.mp3  and  voice.words.json

# 2. render all three flavours
subreel render voice.mp3 \
  --timings voice.words.json \
  --formats green,webm,mov \
  --out-dir out
# -> out/subs-green.mp4, out/subs-alpha.webm, out/subs-alpha.mov

B. You already have audio — let Whisper transcribe it

subreel render narration.m4a --asr --asr-model small --formats green,webm

The recognised timings are cached to out/narration.words.json and reused on the next run (pass --force-asr to re-run).

C. You already have timings

Any tool that emits the words.json format can drive subreel:

subreel render voice.mp3 --timings my-alignment.json --formats mov

Decoding the transparent WebM from the command line

subs-alpha.webm carries its alpha channel as a VP9 BlockAdditional layer (alpha_mode=1). NLEs and browsers handle this natively, but ffmpeg's native vp9 decoder silently drops that layer, so a plain ffmpeg -i subs-alpha.webm frame.png yields an opaque black frame. Decode with libvpx instead:

ffmpeg -c:v libvpx-vp9 -i subs-alpha.webm -frames:v 1 frame.png   # RGBA, alpha intact

The ProRes .mov needs no such flag — every ffmpeg/ProRes toolchain reads its alpha directly.

Compositing guide

All three outputs are subtitle overlays: they contain only the text (and the audio), so you place them on top of your footage and key out the background.

剪映 (CapCut / JianYing)

1. Import subs-green.mp4 and drop it on a track above your video.
2. Select the clip → 画面 (Picture) → 抠像 (Key) → 色度抠图 (Chroma key).
3. Pick the key colour from the green background — it is 0x00B140 (RGB 0, 177, 64). Use the eyedropper on the green area.
4. Raise 强度 (Strength) until the green is fully gone, then adjust 阴影 (Shadow) to clean up the antialiased edges around the glyphs.

Adobe Premiere Pro

• Green MP4: apply Ultra Key, eyedrop the green, set Matte Generation → Tolerance ~ 30–40 % and Pedestal ~ 10 %.
• Transparent WebM / MOV: just drop subs-alpha.webm (VP9 with alpha) or subs-alpha.mov (ProRes 4444) onto the timeline — no keying needed.

DaVinci Resolve

• Green MP4: add a 3D Keyer node, eyedrop the green, then refine with the Clean Black / Clean White controls.
• Transparent MOV: import subs-alpha.mov and place it on a higher track; Resolve honours the ProRes 4444 alpha directly.

Final Cut Pro

• Green MP4: apply the Keyer effect and pick the green.
• Transparent MOV: add subs-alpha.mov above your footage — the alpha is used automatically.

Any NLE

Transparent WebM (VP9 yuva420p) and transparent MOV (ProRes 4444) can be dragged straight onto the timeline in Premiere, DaVinci and Final Cut with no keying. If your editor is unhappy with VP9 alpha, use the MOV.

words.json timing format

subreel's public interchange format. Times are in seconds (float).

{
  "language": "zh",
  "source": "edge-tts",
  "words": [
    { "text": "你", "start": 0.12, "end": 0.31 },
    { "text": "好", "start": 0.31, "end": 0.58 }
  ]
}

Validation rules (enforced on load, with a clear error otherwise):

• words must be present and non-empty;
• every word needs text (string) and numeric start / end;
• start < end for every word;
• start values are monotonically non-decreasing (equal is allowed, so two words may share a boundary).

CLI reference

subreel [-h] [--version] [-v] [-q] {tts,render} ...

Global options: --version, -v/--verbose (INFO logging), -q/--quiet.

subreel tts

subreel tts (--text FILE | --text-inline STR) [--voice VOICE]
            [--rate RATE] [--pitch PITCH] --out AUDIO [--timings JSON]

subreel render

subreel render AUDIO [--timings JSON] [--asr] [--asr-model M] [--language L]
               [--force-asr] [--formats F] [--out-dir DIR] [--size WxH]
               [--fps N] [--green HEX] [--no-audio] [--fontsdir DIR]
               [--font NAME] [--fontsize N] [--align POS] [--highlight HEX]
               [--color HEX] [--no-fade] [--max-chars N] [--max-lines N]
               [--hold SEC]

Exit codes: 0 on success, 2 for every recoverable error (missing ffmpeg, missing audio, no timing source, bad words.json, ffmpeg failure).

Development

git clone https://github.com/paxoscn/subreel.git
cd subreel
pip install -e ".[dev,tts,asr]"

# run the offline test suite (no network, no model downloads)
python3 -m pytest tests/ -q

# byte-compile check
python3 -m compileall src/

The tests mock edge-tts, faster-whisper and ffmpeg, so they run anywhere.

CI (.github/workflows/ci.yml) runs the suite on Python 3.9–3.13, then builds the sdist + wheel, twine check --stricts the metadata and smoke-installs the wheel. Release automation (tag → PyPI via OIDC trusted publishing) is documented in docs/releasing.md.

License

Apache-2.0 © 2026 paxoscn

Credits

subreel stands on the shoulders of these projects:

• ffmpeg — probing, filtering and encoding;
• libass — ASS subtitle rendering (via ffmpeg's ass filter);
• edge-tts — neural text-to-speech and word-boundary timings;
• faster-whisper — fast word-level speech recognition.