srt2speech 1.0.0

Synthesize a timestamp-synced speech track from a subtitle file and mux it into video

srt2speech

Turn a subtitle file into a timestamp-synced speech track and mux it into a video.

Give it a video + an .srt (or .vtt/.ass); it synthesizes audio where each subtitle is spoken at its timestamp, then optionally muxes the track back in with ffmpeg. Useful for restoring lost audio, rough translation dubs, narrating silent videos, or adding audio description by reading only the descriptive/SDH cues.

It does the SRT→audio part well and nothing else: no translation, no transcription — bring an already-final subtitle file.

Requirements

• Python ≥ 3.11, uv
• ffmpeg / ffprobe on PATH
• A TTS backend:
  • piper — a local gopipertts server (free, default; set SRT2SPEECH_PIPER_URL if not on http://localhost:8080)
  • openai — gpt-4o-mini-tts (set OPENAI_API_KEY)
  • elevenlabs — eleven_multilingual_v2 (set ELEVENLABS_API_KEY)

Install

uv sync

Usage

# generate a synced track with the local piper backend, sized to the video
uv run srt2speech generate subs.srt --video clip.mp4 -o track.wav

# generate + mux into the video in one step
uv run srt2speech run clip.mp4 subs.srt -o dubbed.mp4

# paid backend with delivery guidance
OPENAI_API_KEY=... uv run srt2speech generate subs.srt \
    --backend openai --voice coral --instructions "calm documentary narration" -o track.wav

# audio description: only descriptive/SDH cues, mixed over the existing audio
uv run srt2speech run movie.mkv subs.srt --mode descriptive --mux-mode mix -o described.mkv

# mux an existing track yourself
uv run srt2speech mux clip.mp4 track.wav -o dubbed.mp4

# list a backend's voices
uv run srt2speech voices --backend openai

Docker Compose

Runs a local piper server plus an on-demand CLI; no host Python or ffmpeg needed. Put your video and subtitles in ./data (mounted at /data); pulled voices are cached in ./voices.

# 1. start the piper TTS server (preloads the default voice)
docker compose up -d gopipertts

# 2. run the CLI against files in ./data
docker compose run --rm srt2speech run /data/clip.mp4 /data/subs.srt -o /data/dubbed.mp4

# 3. tear down when done
docker compose down

For the OpenAI backend, put OPENAI_API_KEY=sk-... in a .env file (gitignored) — Compose loads it automatically and passes it through to the CLI container.

Sync strategies (--strategy)

Speech rarely fits a cue's window exactly. The fit engine offers:

• hybrid (default) — fit into the cue window plus the silent gap before the next cue; only then speed up, capped by --max-speedup (default 1.15).
• overflow — never speed up; let speech run into following silence (best quality, can drift).
• precise — fit the exact cue window, speeding up to the cap.

Modes (--mode)

all (default) · descriptive (SDH/audio-description only) · dialogue (drop sound cues).

Development

uv run pytest
uv run ruff check