screencast-narrator 0.6.0

Turn raw screen recordings + narration text into polished narrated screencasts

screencast-narrator

Turn raw screen recordings + narration text into polished narrated screencasts.

screencast-narrator is a Python library and CLI that takes a browser screen recording together with a narration timeline and produces a final video with:

• Text-to-speech narration synced to on-screen actions
• QR-code-based frame-accurate synchronization
• Automatic freeze-frame insertion when narration overflows action duration
• Dead-air gap detection and cutting
• Interactive HTML timeline visualization

Installation

pip install screencast-narrator

For TTS support (Kokoro):

pip install screencast-narrator[tts]

For TTS support (Gemini, requires GEMINI_API_KEY or GOOGLE_API_KEY):

pip install screencast-narrator[gemini]

System dependencies:

# macOS
brew install ffmpeg zbar

# Ubuntu/Debian
apt-get install ffmpeg libzbar0

# Windows
winget install GyanDev.FFmpeg
# The pyzbar Python package ships its own libzbar DLL but needs the Visual C++ 2013 runtime:
winget install Microsoft.VCRedist.2013.x64

Quick Start

As a library with Playwright

from pathlib import Path
from playwright.sync_api import sync_playwright
from screencast_narrator.storyboard import Storyboard
from screencast_narrator.sync_frames import inject_sync_frame
from screencast_narrator.merge import process

output_dir = Path("my-screencast")
storyboard = Storyboard(output_dir)

with sync_playwright() as p:
    browser = p.chromium.launch()
    context = browser.new_context(
        record_video_dir=str(output_dir / "videos"),
        record_video_size={"width": 1280, "height": 720},
    )
    page = context.new_page()

    # Narration bracket with sync frames
    nid = storyboard.begin_narration("We open the example website.")
    inject_sync_frame(page, nid, "START")
    storyboard.add_screen_action("Navigate to example.com")
    page.goto("https://example.com")
    inject_sync_frame(page, nid, "END")
    storyboard.end_narration()

    context.close()
    browser.close()

# Produce the final narrated video
process(output_dir)

As a CLI

screencast-narrator /path/to/recording-output/

# Options:
#   --offline                          use the offline Kokoro TTS backend
#   --tts-backend kokoro|edge|gemini   select the TTS backend explicitly
#                                      (gemini reads GEMINI_API_KEY or GOOGLE_API_KEY)
#   --debug-overlay                    burn narration timing overlay into the video
#   --font-size N                      debug overlay font size

The directory must contain:

• A storyboard.json file (produced by Storyboard or hand-written)
• A video file (.webm) in a videos/ subdirectory with sync frames embedded

API

The screencast-narrator API is JSON-based and language-agnostic. Any browser automation framework that can record video and execute JavaScript can produce the inputs.

See docs/api.md for:

• Timeline JSON schema
• Sync frame protocol specification
• Sample code in Python, Java, and TypeScript
• Pipeline processing details

Architecture

The pipeline has these stages:

1. Storyboard (storyboard.py) — Declares narration text and screen actions. No timestamps — timing comes from sync frames.

2. Sync frame injection (sync_frames.py) — Inject green QR-code overlay frames into the browser at narration bracket boundaries for frame-accurate sync.

3. TTS generation (tts.py) — Convert narration text to speech audio files. Pluggable backend; ships with Kokoro TTS.

4. Sync detection (sync_detect.py) — Extract frames from the recorded video, detect green sync frames, decode QR codes to map video frames to narration events.

5. Freeze frame calculation (freeze_frames.py) — When narration audio is longer than the on-screen action, calculate where to insert freeze frames so audio and video stay in sync.

6. Video merge (merge.py) — Orchestrate FFmpeg to build the final video: strip sync frames, insert freeze frames, overlay audio, cut dead air gaps.

7. Timeline visualization (timeline_html.py) — Generate an interactive HTML timeline showing bracket positions, freeze frames, gap cuts, and audio durations.

Custom TTS Backend

Subclass TTSBackend:

from pathlib import Path

from screencast_narrator.tts import TTSBackend

class MyTTS(TTSBackend):
    def resolve_voice(self, voice: str) -> str:
        # Map a logical voice name (female-1..4, male-1..4) to a backend voice ID
        ...

    def _generate_raw(self, text: str, output_path: Path, voice: str) -> None:
        # Generate a WAV file at output_path
        ...

# Use it — the factory receives the storyboard language
from screencast_narrator.merge import process
process(target_dir, tts_backend_factory=lambda language: MyTTS(language=language))

Development

git clone https://github.com/mmarinschek/screencast-narrator.git
cd screencast-narrator
pip install -e ".[dev,tts]"

# Run tests
pytest tests/ -v

# On macOS, if pyzbar can't find libzbar:
DYLD_LIBRARY_PATH=/opt/homebrew/lib pytest tests/ -v

License

Apache License 2.0 — see LICENSE.