🎬 Shorts Factory

Turn one long video into a batch of ready-to-post 9:16 viral shorts — locally, free, offline.

long video ──▶ transcribe ──▶ pick highlight moments ──▶ reframe to 9:16 ──▶ burn karaoke captions ──▶ short_01.mp4 … short_05.mp4 + report

Features

• 🗣 Transcription — faster-whisper (offline, free) with word-level timestamps; cached to JSON so re-runs are instant.
• 🎯 Highlight picking — heuristic virality scoring: question hooks, numbers, power words, speech pace, clean pause boundaries. Optionally an LLM re-ranks candidates and writes titles + hashtags (set OPENAI_API_KEY).
• 📱 Smart 9:16 reframe — face detection (OpenCV) picks the speaker and zooms/crops to 1080×1920; falls back to center crop. Loudness normalized to −14 LUFS.
• 🎨 Karaoke captions — CapCut-style bold uppercase captions with the active word highlighted, burned in via ffmpeg ASS.
• 🖼 Auto thumbnails — per clip it samples candidate frames, scores them on sharpness/exposure/face presence (the same face detector the reframe uses) and stamps the hook title + hashtags onto a 1080×1920 JPEG (thumb_01.jpg …).
• 📊 Report — report.json + report.md with time ranges, titles, hashtags, scores, transcripts.
• 🌐 Input — local files or YouTube/URL links (via yt-dlp).

Install

Requirements: Python 3.11+, FFmpeg on PATH.

# from a source checkout (recommended for now)
git clone <your-repo-url> shorts-factory
cd shorts-factory
pip install -r requirements.txt
pip install -e .          # adds the `shorts-factory` command

# or once published to PyPI:
# pip install shorts-factory

Usage

Web UI (recommended)

cd D:\Cline\shorts-factory
python -m shorts_factory ui

CLI

cd D:\Cline\shorts-factory

# 5 clips from a local file
python -m shorts_factory run "C:\Videos\podcast.mp4"

# from YouTube, 3 clips, better accuracy, custom output dir
python -m shorts_factory run "https://youtu.be/XXXX" --top 3 --model small -o D:\outputs

# options
python -m shorts_factory run --help

Useful flags: --top N, --model tiny|base|small|medium, --min-dur 12, --max-dur 45, --language en, --no-llm, --no-captions, --no-thumbnails, --thumbnail-font <ttf>, --font "Impact", --no-uppercase, --no-loudnorm.

Optional: GPT-powered picks & titles

$env:OPENAI_API_KEY = "sk-…"
python -m shorts_factory run video.mp4 --llm

Any OpenAI-compatible endpoint works via OPENAI_BASE_URL / SF_LLM_MODEL.

Output layout

shorts_output/<video name>/
├── short_01.mp4 … short_05.mp4   # 1080×1920, captions burned, loudness-normalized
├── thumb_01.jpg … thumb_05.jpg   # 1080×1920 cover frame with the hook title baked in
├── report.json / report.md       # titles, hashtags, time ranges, transcripts
├── cache/*.transcript.json       # reused on re-runs
└── work/clip_XX/subs.ass         # generated caption files

How highlight scoring works

Candidates are built from sentence-like transcript units (pauses > 0.7 s + punctuation), sized 12–45 s, then scored on: question openers, concrete numbers/$/%, power words, speaking pace (2.2–4.6 words/s), clean pause boundaries, ideal length. Overlapping picks are removed (non-max suppression). With OPENAI_API_KEY, a GPT pass re-ranks and writes click-worthy titles + hashtags.

Tests

python -m pytest                        # unit tests (captions, scoring, reframe, transcript, utils)
python tests\karaoke_probe.py           # verifies caption highlight colors against libass
python tests\make_test_video.py         # builds a ~90 s speech demo video (needs network)
python -m shorts_factory run tests\data\test_input.mp4 --top 2   # end-to-end smoke test

Limitations / roadmap

• Single frontal speaker assumed for reframing (multi-speaker tracking = future work).
• Heuristic highlight picks are good but not semantic — enable the LLM option for best results.
• CPU transcription: base ≈ realtime, small more accurate but slower; GPU supported by faster-whisper config.
• Ideas next: auto B-roll from transcript keywords, split-screen layout for gameplay footage, TikTok-style zoom punch-ins, direct upload.

License

MIT — see LICENSE.