CaptionForge

English | Español

Local, free automatic video captions - the CapCut/Kapwing auto-caption experience, but 100% on your own machine. No watermark, no monthly limit, no account.

Drag in a video, get back a .srt file and/or the video with subtitles burned in, in a modern social-media style. Transcription runs locally via faster-whisper; translation (to any language, not just English) runs locally via argos-translate; burning subtitles into the video uses ffmpeg. Nothing leaves your machine.

Requirements

• Python 3.10+
• ffmpeg on your PATH

Install & run

pip install -e ".[dev]"
captionforge serve

This starts a local server (default http://127.0.0.1:8420/) and opens it in your browser. The ES / EN switch in the top-right corner sets the UI language (remembered for next time via localStorage; it defaults to your browser's own language). Drag in a video, choose a Whisper model size and optionally a source language / translation target, and click Generate captions.

Once transcription finishes:

• Download .srt, .vtt, or .ass directly - the same segments, three formats (.vtt for a plain HTML <video><track>, .ass for an editor that wants real styling/karaoke tags).
• Edit captions - fix a transcription mistake before burning, and drag each segment's start/end directly over a waveform to retime it. Words faster-whisper transcribed with low confidence are underlined (with a subtle highlight) right in the editor, so you know what to double-check instead of trusting the transcript blindly.
• Pick a caption style (Modern, TikTok bold, YouTube classic, Minimal) and turn on word-by-word karaoke highlighting for the burn - available whenever a segment has per-word timing, whether that's faster-whisper's own real timing or the approximate, character-length-based timing this app synthesizes after you translate or edit a segment's text (see "Known limitations").
• Burn into video - a separate, on-demand step from transcription - you're never forced to re-encode the whole video just to get the text.

Recent jobs, below the main card, remembers your last 10 uploads in this browser (localStorage) with direct re-download links for all four formats - handy after you've moved on to a new video and the "current job" indicator above has moved with you.

CaptionForge processes one video at a time by design on the backend - a second upload while one is running is rejected with a clear error rather than silently overwritten. The frontend builds on that: select or drop several videos at once and they're queued client-side (name + status: waiting/running/done/error), uploaded one at a time, automatically moving to the next file as soon as the current one reaches done or error - so one bad file in a batch doesn't block the rest. Editing and re-burning are only offered for the current job (the last one in a queued batch); older jobs in history are downloads only (see "Known limitations").

Architecture

The upload request returns in milliseconds - the real work runs as a background task while the browser watches it happen over a live SSE stream, not by polling.

How it's built

• src/captionforge/srt.py - pure formatting/assembly for .srt, .vtt, and karaoke-capable .ass (per-word \k tags whenever Segment.words is present). WordTiming.probability carries faster-whisper's own per-word confidence (None only for a word this app synthesized itself, never a real transcription). redistribute_word_timings() is the shared approximate-timing heuristic both translation and text-editing use once a segment's words no longer match its ORIGINAL per-word timing - see "Known limitations" for exactly what it does and doesn't guarantee. Also the plain-dict (de)serialization used to persist segments to segments.json. No I/O.
• src/captionforge/translate.py - local translation of already-timed segments via argos-translate, decoupled from Whisper (whose own task="translate" only ever translates into English). The ORIGINAL-language per-word timing can't survive a translation (different words, count, and often order) - rather than dropping word-level timing outright, redistribute_word_timings() approximates new timing for the translated text.
• src/captionforge/waveform.py + ffmpeg_utils.build_waveform_extract_cmd
  • downsampled audio amplitude data for the editor's waveform backdrop: a single ffmpeg pass decodes+resamples a job's original video to raw 8-bit PCM at a low, fixed sample rate, bucketed down to at most 2000 peaks before being sent to the browser.
• src/captionforge/ffmpeg_utils.py - pure ffmpeg argv construction (audio extraction, subtitle burning, waveform extraction) - never executes anything itself. STYLE_PRESETS (modern/tiktok/youtube/minimal) is the single source of truth both the plain-SRT force_style burn and the karaoke .ass burn render from.
• src/captionforge/jobs.py - an in-memory, thread-safe job state machine (queued -> extracting_audio -> transcribing -> done -> burning_subtitles -> burned, or error from anywhere). Holds ONE job at a time by design - segments.json and the output files persisted to disk (not this in-memory store) are what let "recent jobs" history keep working after a newer job takes over.
• src/captionforge/pipeline.py - orchestrates the above: ffmpeg runs via asyncio.create_subprocess_exec, Whisper/Argos (blocking, CPU-bound) run in a worker thread via asyncio.to_thread, so the server stays responsive (including the live progress stream) while a video is processing. Writes segments.json alongside the .srt; the karaoke burn path builds a karaoke.ass from it on demand.
• src/captionforge/app.py + routes/ - the FastAPI layer: upload, Server-Sent Events for live progress, the .srt/.vtt/.ass/video downloads (with a disk-existence fallback for a job that's no longer the one JobStore is tracking - safe because CaptionForge's one-job-at-a-time design guarantees any older job already reached a terminal state), segment editing (GET/PUT .../segments, current job only - PUT accepts text, and/or start/end for a waveform-drag retime), GET .../waveform (works for any job, current or historical - it only ever needs the original video file), and burn (style/karaoke form fields).
• src/captionforge/static/ - the frontend: one plain HTML/CSS/JS page, no build step, no framework. i18n.js is a small flat-dictionary translator (Spanish/English, localStorage-backed) that drives every data-i18n-tagged element in index.html; job stage labels are derived client-side from the language-neutral status field the API already returns, not from the backend's own (Spanish-only) stage_label text. The segment editor (app.js) renders the waveform as a <canvas> with draggable start/end handles per segment, and underlines any word below a confidence threshold using the per-word probability the API returns.

scripts/smoke_test_pipeline.py exercises the whole transcribe -> translate -> burn pipeline directly against a real video, no server involved - the fastest way to sanity-check the core after touching anything Whisper/ffmpeg/Argos-related.

Development

python -m venv .venv
source .venv/Scripts/activate   # or .venv/bin/activate on Linux/macOS
pip install -e ".[dev]"
pytest

tests/fixtures/tiny_test_clip.mp4 is a real ~10s clip (synthesized speech) used by the live-pipeline tests - not a mock.

Packaged build (first step)

A deliberately narrow first step toward issue #2's packaged installer - not the full multi-platform, signed, GPU-aware installer described in "Roadmap" below.

pip install -e ".[build]"
python scripts/build_installer.py

This produces a single dist/captionforge executable (.exe on Windows) via PyInstaller, driven by scripts/captionforge.spec (the commented, reproducible build config) from the scripts/pyinstaller_entrypoint.py entry point - the same thing captionforge serve does (open the browser, serve on the default port with the default model), minus argparse, since a packaged executable has no terminal to pass flags to.

What this covers:

• One platform at a time - whichever OS you run the build script on. PyInstaller doesn't cross-compile; a Windows machine only ever produces a Windows executable, same for Linux/macOS.
• CPU-only - bundles whatever faster-whisper/ctranslate2 backend is already installed in the build venv, no CUDA/cuDNN.

What this deliberately does not cover yet (see issue #2's own discussion for why each of these is a separate, non-trivial piece of work):

• Multiple platforms from one place - building/distributing Windows + macOS + Linux together.
• Code signing - the built executable will trigger Windows SmartScreen / macOS Gatekeeper warnings on first run.
• GPU/CUDA build selection - no per-OS GPU detection or a GPU-enabled build variant.
• Bundling ffmpeg - the built executable still expects ffmpeg on PATH, exactly like running from source.

Known limitations

• argos-translate's default compute_type="auto" resolves to a quantized kernel that silently produces garbage (repetition-loop) output for at least one language pair on at least one real CPU - verified live during development. captionforge forces float32 (see translate.py) to avoid this; if you use argos-translate directly elsewhere, verify your own language pair isn't affected before trusting quantized output.
• The UI language switch is frontend-only. Everyday status text (stage labels, generic error framing) is fully bilingual, but the rare server-generated error message - an unsupported file format, a job conflict, an ffmpeg failure - is still written in Spanish by the backend and shown as-is regardless of the selected UI language.
• Editing captions and re-burning are only available for the CURRENT job - once a new upload starts, JobStore forgets the old one (by design; see jobs.py), so an older job in "recent jobs" offers downloads only. This matches the natural flow (transcribe -> optionally edit -> burn) and the one-job-at-a-time state machine, which has no path back from BURNED.
• Karaoke highlighting needs word-level timing. A segment translated or manually edited gets APPROXIMATE word timing instead of its original (real) one: redistribute_word_timings() splits the segment's existing [start, end) span across the new text's words, proportionally by character length - a cheap, honest stand-in for real forced alignment, not an acoustically verified one. It is NOT lip-synced: a translated sentence's words rarely land where the corresponding sound actually occurs, especially for language pairs with very different word order. Every word this app synthesizes this way has probability: null in the segments API response specifically so nothing mistakes it for a real transcription confidence score. The karaoke checkbox is simply hidden when no segment has any word timing at all (real or approximate). Real forced alignment (a wav2vec2-style model, the way WhisperX does it) would fix this properly, at the cost of a whole new model dependency - out of scope for now; see the diarization/voice-separation issues below for the same "new heavy ML dependency" trade-off applied to two other features.
• The low-confidence word highlighting in the editor is only as good as faster-whisper's own per-word probability - a word can be confidently wrong (misheard but pronounced clearly) or unconfidently right (correct despite noisy audio). Treat the underline as "worth a second look", not as a correctness guarantee.
• The waveform editor's drag handles let you shrink or grow a segment freely; there's no validation against a NEIGHBORING segment's start/end, so it's possible to drag two segments into overlapping (or gapped) time ranges. Nothing crashes, but review the result before burning if you make a large adjustment.
• "Recent jobs" lives in localStorage, so it's private to one browser - it does not survive clearing site data and is never shared between devices.
• The upload queue lives only in the page's memory - reloading mid-batch resumes the single file that was actively uploading (same as any single job), but any files still queued behind it are lost; re-select them to keep going.
• A job's files (video, .srt/.vtt/.ass, segments.json) are deleted automatically 7 days after they were last written - each new upload prunes anything past that age. An entry can outlive its files in "recent jobs" (which has no expiry of its own); its download links just 404 once that happens.

Roadmap

Ideas worth doing eventually, deliberately not started yet:

• A packaged native installer (Windows .exe, macOS .dmg, Linux .AppImage/.deb) so a user doesn't need Python or ffmpeg pre-installed. A first, narrow step exists today (see "Packaged build (first step)" above: one platform, CPU-only, unsigned, ffmpeg not bundled). What's still missing - multi-platform distribution from one place, a bundled static ffmpeg per OS, per-OS GPU/CUDA detection, and code-signing (to avoid Windows SmartScreen / macOS Gatekeeper warnings) - is comparable in effort to building the app itself, which is why it's out of v1 on purpose.
• A hosted version - CaptionForge needs real CPU (or GPU) for Whisper/ffmpeg, so a free-tier host isn't enough for serious use; a paid host is the realistic next step if there's ever demand for a "no-install-at-all" option. See "Support this project" below.
• Speaker diarization ("who said what") via pyannote.audio - see issue #4 for why it's deferred: a second heavy PyTorch-based ML dependency, plus real friction from pyannote's gated Hugging Face models (an account + accepted terms + a personal access token, unlike faster-whisper's anonymous downloads today).
• Vocal/source separation before transcription (via Demucs) for noisy or music-heavy audio - see issue #5 for why it's deferred: a third heavy ML dependency, real added runtime cost for the common case (clean dialogue) that doesn't need it, and a quality trade-off that needs real before/after comparison, not just an assumption that separation always helps.
• Real forced alignment after a translation or manual text edit (a wav2vec2-style model, the way WhisperX does it) - today's approximate, character-length-based word-timing redistribution (see "Known limitations") is a deliberately cheap stand-in for this, not a replacement for it.
• A real parallel backend queue (processing more than one video at once) - CaptionForge is one-job-at-a-time by design today (see jobs.py and the frontend's own client-side upload queue, which uploads sequentially specifically because the backend can only run one job at a time). Worth doing eventually for a multi-core machine, but a genuinely bigger change (worker pool, per-job resource limits, a queue that survives a server restart) than anything else in this list - no concrete plan yet, flagged here only so it isn't mistaken for an oversight.

Support this project

CaptionForge is free and local by design, and will stay that way. A tip doesn't unlock anything - it goes toward eventually paying for a real host, so people who don't want to install anything have a "no-install-at-all" option too:

• Ko-fi: ko-fi.com/bryandero98
• USDT (TRC20): TEG4Kk2qXYMQ4mHNd7dPhSPRyT14CGr2or - double-check the network is set to TRC20 before sending; a transfer on the wrong network can't be recovered.

Ideas & contributions

Suggestions for what CaptionForge should do next are welcome, not just bug reports - open an issue with what you'd want, even a rough one. See CONTRIBUTING.md for how to send a PR.