video-caption-ai

video-caption-ai turns video audio into SRT subtitles with faster-whisper, optionally refines subtitles with an OpenAI-compatible AI service, and can mux or burn subtitles back into video.

Features

• FFmpeg audio extraction
• faster-whisper transcription
• AI semantic segmentation
• AI transcript correction with validation and retries
• Optional TXT/Markdown reference transcript matching
• SmartSub-inspired timing and readability guards
• Soft subtitle muxing or hard subtitle burning
• Hard subtitle style presets: classic, yellow, cinema, top

Requirements

• Python 3.10 or newer
• FFmpeg available as ffmpeg on PATH
• CPU or NVIDIA CUDA runtime as appropriate for faster-whisper

Install for development

python -m pip install -e ".[dev]"

The package provides the video-caption command. video-subtitle remains as a compatibility alias. It can also be run with:

python -m video_caption_ai --help

Transcribe video

video-caption input.mp4 output.srt --model small --language zh --device cpu --no-vad

Use a reference transcript without changing Whisper cue timing:

video-caption input.mp4 output.srt `
  --model small --language zh --device cpu --no-vad `
  --reference-script reference.md

The reference file can be .txt or .md. Markdown headings, lists, links, emphasis, and fenced code are cleaned before matching. If the text does not match the ASR transcript reliably, the original ASR cue text is kept.

AI refinement

The AI endpoint must expose an OpenAI-compatible /chat/completions endpoint.

video-caption input.mp4 output.srt `
  --model small --language zh --device auto --no-vad `
  --ai-segmentation --ai-correction `
  --ai-base-url "http://192.168.0.41:9292/v1" `
  --ai-model "Qwen3.6-35b-A3B-UD-Q6_K_XL-MTP"

An API key is optional for local services.

Compose an existing SRT

Soft subtitles, selectable in a compatible player:

video-caption input.mp4 existing.srt --compose-only `
  --video-output subtitled.mkv --subtitle-mode soft

Hard subtitles burned into the picture:

video-caption input.mp4 existing.srt --compose-only `
  --video-output subtitled.mp4 --subtitle-mode hard `
  --style-preset yellow

Override a preset when needed:

video-caption input.mp4 existing.srt --compose-only `
  --video-output subtitled.mp4 --subtitle-mode hard `
  --style-preset yellow --font-size 56 --outline-width 4

Python API

from video_caption_ai import compose_video, transcribe_video

# CLI-oriented transcription entry point is available through video-caption.
# Video composition is also available as a Python function.

Development checks

python -m pytest
python -m video_caption_ai --help
python -m build

License

MIT. See LICENSE.

Credits

video-caption-ai is an independent Python rewrite inspired by and based on the subtitle transcription and refinement workflow of SmartSub. It is not the official SmartSub distribution. The original SmartSub project is licensed separately; see its repository for its license and attribution details.