rot

rot is a Python library for assembling high-energy vertical videos for Instagram Reels, YouTube Shorts, and TikTok. It combines backgrounds, dialogue, local TTS, synced captions, speaker portraits, overlays, transitions, and effects into one FFmpeg render.

The default preset produces a 1080×1920 MP4 with H.264 video, AAC 48 kHz stereo audio, constant 30 fps, a 10 Mbps target/12 Mbps ceiling, yuv420p, and SDR Rec.709 metadata. The encoder is also given an 8 Mbps minimum rate target.

Project map

• Documentation home is the GitHub Pages-friendly guide to installation, API workflows, and references.
• Recipes provides copyable projects for narration, rankings, and clip discovery.
• Architecture explains the render pipeline and extension boundaries.

Requirements

• Python 3.12 or newer
• uv
• A system FFmpeg build containing FFprobe, libx264, AAC, and libass

On Debian or Ubuntu:

sudo apt-get install ffmpeg
uv sync --group dev
uv run rot doctor

Install only the integrations you use:

uv sync --extra chatterbox   # Chatterbox (`tts` remains an alias)
uv sync --extra kokoro       # Kokoro-82M
uv sync --extra align        # Stable-TS word alignment
uv sync --extra openrouter   # OpenRouter script parsing
uv sync --extra youtube      # YouTube downloading with yt-dlp
uv sync --extra twitch       # Official Twitch clip downloading
uv sync --extra publish      # Official YouTube, Instagram, and TikTok publishing APIs

Quick start

Create script.rot:

@alex [id=hook]: You will not believe what happened next.
@sam: There is absolutely no way.
@alex [audio=recordings/final-line.wav]: Look at this.

Create video.py:

from rot import ChatterboxVoice, Project, StableTSAligner

project = (
    Project.short_form()
    .background("assets/gameplay.mp4", trim=(12, 42), loop=True)
    .add_speaker(
        "alex",
        voice=ChatterboxVoice("assets/alex-reference.wav"),
        portrait="assets/alex.png",
    )
    .add_speaker(
        "sam",
        voice=ChatterboxVoice("assets/sam-reference.wav"),
        portrait="assets/sam.png",
        portrait_position="bottom-left",
    )
    .script_file("script.rot")
    .captions("pop")
    .overlay_image("assets/reaction.png", during="hook", animation="bounce")
    .soundtrack(
        "assets/music.mp3",
        volume=0.12,
        trim=(8, 28),
        fade_in=0.5,
        fade_out=0.8,
        ducking=True,
    )
    .with_aligner(StableTSAligner("base"))
)

Render it:

uv run rot render video.py -o short.mp4

Project files are trusted Python code. Use video.py:another_project to select an object other than the default project.

Composition

Backgrounds accept videos or still images and use cover fitting by default. Add multiple clips with add_clip, then select cut, fade, crossfade, slide-left, slide-right, or zoom between them. Video effects include zoom, punch zoom, pan, shake, blur, grayscale, and saturation.

A single still image automatically fills the dialogue duration. Without dialogue, or when a still is one item in a multi-clip timeline, give it an explicit duration. Stills cannot be trimmed or have their playback speed changed:

project = (
    Project.short_form()
    .background("title-card.png", duration=1.5, fit="contain", fill="blur")
    .add_clip("gameplay.mp4", trim=(12, 25), loop=False)
)

For horizontal footage, fit="custom" provides a controllable middle ground between preserving the complete frame and filling the vertical canvas. fit_amount=0.0 is equivalent to contain, while fit_amount=1.0 is equivalent to cover:

project.add_clip(
    "horizontal.mp4",
    fit="custom",
    fit_amount=0.4,
    fill="blur",
    fill_blur=40,
    anchor="center",
)

Intermediate values enlarge the clip without distortion, crop the overflow according to anchor, and pad any remaining uncovered canvas area. The default fill="black" uses solid letterboxing; fill="blur" places a blurred, full-canvas copy of the clip behind the sharp foreground.

Streamer footage can extract an embedded facecam from the same custom-fit source. Both rectangles use normalized 0–1 coordinates, so the layout survives resolution changes. The facecam preserves its aspect ratio and cover-fills its destination:

from rot import Facecam, NormalizedRect

project.background(
    "stream.mp4",
    fit="custom",
    fit_amount=0.35,
    anchor="top",
    facecam=Facecam(
        crop=NormalizedRect(x=0.02, y=0.04, width=0.24, height=0.32),
        destination=NormalizedRect(x=0.1, y=0.7, width=0.8, height=0.25),
    ),
)

The default black fill leaves the remaining canvas clean. Set fill="blur" explicitly to retain blur behind both the fitted clip and extracted facecam.

Use normalized focus=(x, y) to choose the exact source point retained by cover or custom cropping. For contain or custom, position=Placement(...) independently places the fitted foreground within the output canvas:

project.background(
    "gameplay.mp4",
    fit="custom",
    fit_amount=0.35,
    focus=(0.72, 0.4),
    position=Placement(0.5, 0.08, anchor="top"),
)

Both controls clamp safely to the available source/canvas bounds. Omit either one to retain the corresponding behavior from the legacy anchor option.

project = (
    Project.short_form()
    .background("one.mp4", trim=(2, 8), loop=False)
    .transition("crossfade", duration=0.25)
    .add_clip("two.mp4", trim=(4, 12), keep_audio=True, volume=0.25)
    .effect("saturation", amount=1.3)
    .soundtrack("music.mp3", volume=0.12)
)

Use overlay_image(..., at=3, duration=2), during="line-id", or speaker="alex" to bind an overlay to time. A registered speaker portrait automatically follows that speaker's utterances.

Image overlays also accept during_clip="clip-id" or a zero-based clip index. An absolute image without duration uses the reaction-friendly two-second default. PNG alpha is preserved; JPEG, PNG, and static WebP work when the installed FFmpeg can decode them.

soundtrack configures one background-music bed. The selected trim repeats by default; disable looping to play it once, use fades for clean boundaries, and opt into smooth dialogue ducking:

project.soundtrack(
    "music.mp3",
    volume=0.12,
    trim=(15, 45),
    loop=True,
    fade_in=0.4,
    fade_out=0.8,
    ducking=True,
)

A later soundtrack call replaces the previous one, and music never changes video duration.

Non-caption text uses overlay_text. Assign clips stable IDs and bind a title to each complete clip without calculating timestamps:

project = (
    Project.short_form()
    .background("number-5.mp4", clip_id="rank-5", keep_audio=True, loop=False)
    .add_clip("number-4.mp4", clip_id="rank-4", keep_audio=True, loop=False)
    .overlay_text("#5 — Huge comeback", during_clip="rank-5", position="top")
    .overlay_text("#4 — Impossible save", during_clip="rank-4", position="top")
)

Every layered element also accepts a normalized Placement. Its anchor selects which point on the element is attached to (x, y); existing named positions remain available:

from rot import Placement

project.overlay_text(
    "[color=#FFE135]#5[/color] — [i]Huge comeback[/i]",
    during_clip="rank-5",
    position=Placement(0.5, 0.08, anchor="top"),
)
project.overlay_image(
    "assets/reaction.png",
    during_clip="rank-5",
    position=Placement(0.92, 0.82, anchor="bottom-right"),
)

Safe inline tags are [color=#RGB], [color=#RRGGBB], [b], [i], [u], [font=...], and [size=...]; tags may nest, and doubled brackets produce literal brackets. The same syntax works inside dialogue captions. Formatting is stripped before speech generation, alignment, and SRT output. During captions, the active-word highlight temporarily wins over an inline color.

Clips render in the order they are added. during_clip accepts either a clip ID or a zero-based clip index. Text can also use at/duration, during="line-id", or speaker="alex"; an at overlay without a duration remains visible through the end of the video. Text overlays render even when RenderSettings(captions=False). With a transition, the outgoing title changes to the incoming title at the transition midpoint. See the ranked countdown recipe for a complete example.

Captions and voices

Caption presets are classic, pop, karaoke, and bounce. Pass a CaptionTheme for full font, color, outline, safe-area, casing, and word-group control. The built-in renderer writes ASS and burns it with libass; RenderSettings(caption_sidecar=True) also emits SRT.

Every line may point to prerecorded audio. Otherwise its speaker needs a VoiceProvider such as ChatterboxVoice, KokoroVoice, or a custom provider implementing synthesize. StableTSAligner provides known-transcript word alignment. Without an aligner, rot estimates word timings from the audio duration and emits a warning.

Transcribe speech already inside clips

Opt selected clips into local speech-to-text with transcribe=True. Word timestamps drive the same active-word highlight used by dialogue captions, while a separate top caption lane prevents clip speech from colliding with scripted narration:

from rot import ClipTranscription, StableTSTranscriber

project = (
    Project.short_form()
    .background("stream.mp4", keep_audio=True, transcribe=True)
    .add_clip(
        "interview.mp4",
        keep_audio=True,
        transcribe=ClipTranscription(language="en"),
    )
    .with_transcriber(StableTSTranscriber(model="base"))
    .clip_captions("pop", position=Placement(0.5, 0.08, anchor="top"))
)

transcripts = project.transcribe_clips()

transcribe=True auto-detects language. The default Stable-TS provider requires uv sync --extra transcribe; custom Transcriber implementations remain dependency-free. Transcription is cached by source, trim, speed, language, and provider. It does not alter video duration or implicitly enable source audio. RenderResult.transcripts exposes the same clip-local structured results, and caption_sidecar=True includes clip speech in the SRT output.

Kokoro uses named voices instead of reference-audio cloning and runs well on CPU:

from rot import KokoroVoice

project.add_speaker(
    "alex",
    voice=KokoroVoice("af_heart", speed=1.05),
    language="en-US",
)

KokoroVoice accepts built-in names, comma-separated voice blends, and local .pt voice packs. Set device="cpu", "cuda", or "mps" to override automatic selection. Kokoro produces 24 kHz mono WAV internally; the final render pipeline converts it to the configured AAC 48 kHz stereo output. Use lang_code="a", "b", "e", "f", "h", "i", "j", "p", or "z" to override the language inferred from the speaker. Install the system espeak-ng package for out-of-dictionary English words and languages that use Kokoro's eSpeak phonemizer. Japanese and Mandarin additionally require the corresponding Misaki language extra, for example uv add 'misaki[ja]>=0.9.4' or uv add 'misaki[zh]>=0.9.4'.

Only clone a voice with the represented person's informed permission. rot preserves Chatterbox's generated-audio watermark and provides no watermark-removal feature.

OpenRouter

OpenRouterParser converts free-form text to the same validated script model using strict JSON Schema output. It never runs for normal .rot files and requires an explicit model.

export OPENROUTER_API_KEY=...
uv run rot parse draft.txt --model provider/model --speaker alex --speaker sam -o script.rot

from rot import OpenRouterParser

parser = OpenRouterParser(model="provider/model", speakers=("alex", "sam"))
project.script(free_form_text, parser=parser)

Clip discovery

rot clips TARGET ranks the strongest short-form windows in a YouTube video, an authorized Twitch clip, a local video file, or a whole folder of existing footage.

# A permitted YouTube source (needs the `youtube` extra).
uv run rot clips "https://www.youtube.com/watch?v=VIDEO_ID" \
  --method hybrid --duration 30 --count 5 -o clips

# An existing Twitch clip owned by a channel you broadcast or edit.
export ROT_TWITCH_CLIENT_ID=...
export ROT_TWITCH_ACCESS_TOKEN=...
uv run rot clips "https://clips.twitch.tv/CLIP_ID" \
  --method hybrid --duration 20 --count 2 -o clips

# A local file, or a library ranked across every video in it.
uv run rot clips ./recording.mp4 --duration 20 --count 4 -o clips
uv run rot clips ./gameplay-archive --duration 15 --count 8 --max-per-source 2 -o clips

Folder scans recurse by default, report unreadable files instead of aborting, and cache extracted signals on disk so re-running with a different --count or --duration re-ranks without decoding again (--no-cache to force a fresh pass).

hybrid is the recommended default. It combines visual scene-change strength, frame-to-frame motion, and short-window RMS audio energy, then rejects heavily overlapping results. Use --method scene for edited montages, --method motion for gameplay and action footage that moves constantly without hard cuts, and --method audio for podcasts, interviews, and reactions where energetic speech matters more than cuts. --download-only keeps source.mp4 and reports the suggested time ranges without exporting them.

Ranking is tunable end to end. --scene-weight, --motion-weight, and --audio-weight set the hybrid blend, and every normalization constant is a documented ClipDetectionSettings field. Selected clips snap to a nearby cut or audio trough so they do not begin mid-sentence; pass --no-snap to keep the raw ranked ranges.

The same workflow is available as typed Python APIs:

from rot import ClipDetectionSettings, Project, YouTubeClipFinder

finder = YouTubeClipFinder(
    ClipDetectionSettings(method="hybrid", clip_duration=25, clip_count=3)
)
result = finder.find(
    "https://youtu.be/VIDEO_ID",
    "build/youtube-clips",
)

# Candidates also become trim-aware rot Clip objects without another encode.
project = Project.short_form().background(result.project_clips()[0])

# Each candidate carries the source it came from and its per-signal breakdown.
for candidate in result.candidates:
    print(candidate.source.name, candidate.start, candidate.scene_score, candidate.motion_score)
for warning in result.warnings:
    print(warning)

The exported clips preserve the source dimensions but are accurately cut and encoded as H.264, AAC 48 kHz stereo MP4s. A later Project render applies rot's vertical 1080×1920 output contract. Only download and reuse videos you have permission to process; YouTube availability, age gates, regional restrictions, and authentication are handled by yt-dlp and can still prevent a download. Twitch uses its official Clips Download API and requires a user token with channel:manage:clips or editor:manage:clips; the user must be the broadcaster or an authorized editor for the clip's channel. Pass --twitch-variant portrait only when that clip has an official portrait version available.

Publish a rendered short

Publishing is an explicit step for an existing MP4. It never happens as a side effect of rendering. Install the integration, create OAuth apps with the platforms, and provide current user access tokens through the environment:

uv sync --extra publish
export ROT_YOUTUBE_ACCESS_TOKEN=...
export ROT_INSTAGRAM_ACCESS_TOKEN=...
export ROT_INSTAGRAM_USER_ID=...
export ROT_TIKTOK_ACCESS_TOKEN=...

Keep post metadata—but never tokens—in publish.toml:

[youtube]
title = "The wildest final round"
privacy = "private"
made_for_kids = false
contains_synthetic_media = true
has_paid_product_placement = false
tags = ["shorts", "gaming"]

[instagram]
caption = "The wildest final round #gaming"
share_to_feed = true

[tiktok]
caption = "The wildest final round #gaming"
privacy = "SELF_ONLY"
allow_comments = true
allow_duet = false
allow_stitch = false
brand_organic = false
branded_content = false
ai_generated = true

Preflight all configured accounts, review the destination summary, and publish:

uv run rot publish short.mp4 --config publish.toml

Use --yes only when the command invocation itself represents explicit approval for this post. The command waits for each platform's terminal processing state and reports partial failures without discarding successful remote IDs. See the publishing guide for scopes, account restrictions, the typed Python API, and platform review requirements.

Logging and progress

The library emits records through the rot logger without configuring root logging. render accepts progress=False or a callback receiving ProgressEvent. The CLI displays stage and FFmpeg encoding progress; -v, -vv, and --json-logs control diagnostics.

Outputs are written atomically and existing files are protected unless overwrite=True or --force is supplied. Generated speech is cached under the platform user cache directory.

Development

The complete supported surface—including every class, method, field, callback, parameter, default, and return type—is in the Python API reference.

uv sync --group dev
uv run ruff check .
uv run mypy src/rot
uv run pytest
uv build
uv run twine check dist/*

The optional Chatterbox, Kokoro, and Stable-TS model-download smoke tests are intentionally not part of the ordinary test run.

License

MIT