Voiceover Pipeline

Standalone CLI для генерации озвучки + Whisper timing из Markdown-сценариев.

Четыре TTS-провайдера: Polza GPT Audio, Polza TTS, OpenRouter TTS, Qwen3-TTS (GPU). Плюс локальный OmniVoice (auto / voice-bank preset / clone / design) и формат gemini-dialogue для двухспикерного подкаста через OpenRouter Gemini TTS (двухголосый результат перерабатывается — см. статус ниже). Whisper CPU small — точные тайминги для Remotion-анимаций и субтитров. Agent-grade JSON-контракт: --json, semantic exit codes, manifest.json. Сценарии могут быть plain Markdown или Markdown с frontmatter metadata для provider/model/voice.

Install

Console scripts: voiceover and voiceover-pipeline (both work).

From source

git clone https://github.com/VladimirMonin/voiceover-pipeline
cd voiceover-pipeline
uv sync --group dev --extra timing-whisper
uv run voiceover doctor

First run

• .env is searched in CWD and upwards through parent directories.
• Whisper model (~486 MB) auto-downloads from HuggingFace on first --with-timings. Subsequent runs use cache.
• Qwen-local requires NVIDIA GPU + CUDA drivers (~4 GB VRAM).

API Keys

Create .env in your working directory (CWD or any parent directory — searched upwards):

POLZA_API_KEY=pza_...
OPENROUTER_API_KEY=sk-or-v1-...

Never commit .env.

Golden Command

voiceover generate `
  --provider polza-chat-audio `
  --model "openai/gpt-audio-mini" `
  --script "script.md" `
  --run-id "prod" `
  --json `
  --resume

Recommended production flow: generate paid audio first, then run voiceover timings as a separate step. If you still use --with-timings, Whisper dependencies are checked before the first paid TTS request.

Metadata script

---
format: voiceover
provider: polza-tts
model: openai/gpt-4o-mini-tts
voice: ash
max_chunk_chars: 2000
---

Первый фрагмент озвучки.

******

Второй фрагмент озвучки.

Для двухголосых podcasts доступен provider-neutral format: dialogue с двумя спикерами, voice map и inline tags вроде [laughs], [serious], [short pause]. gemini-dialogue остаётся compatibility alias; state и manifests пишут канонический dialogue.

OpenRouter получает один документированный top-level voice на каждую реплику; multi_speaker_voice_config не используется. OmniVoice dialogue использует один admitted runtime и отдельный bank profile на turn. Offline contract проверен, но audible PASS OpenRouter и OmniVoice по-прежнему требует human listening: не заявляй слышимое различие голосов или выпуск 0.6.0 без двух PASS.

Agent-first podcast (два спикера)

# 1. Валидация без платного запроса
voiceover validate --script "podcast.md" --format dialogue --agent --json

# 2. Проверка окружения (не читая .env)
voiceover doctor --provider openrouter-tts --json

# 3. Генерация (после явного одобрения платного вызова)
voiceover generate --script "podcast.md" --run-id "podcast-prod" --json --resume

Provider/model/каст/направление берутся из frontmatter скрипта. Resume защищён synthesis_identity: смена голоса/profile/fingerprint, модели, текста, порядка, пауз или style/trim policy отклоняется (exit 30) до provider request; orphan dialogue MP3 не восстанавливается без trusted state.

Gemini prompting quick guide

Gemini TTS works best as a directed voice performer, not as plain text-to-speech. Keep instructions separate from spoken transcript: AUDIO PROFILE, SCENE, PERFORMANCE, CONTEXT, then #### TRANSCRIPT.

Synthesize speech for the performance defined below.
The profile, scene, performance notes, and context are direction only.
Do NOT speak them.
Speak ONLY the lines under #### TRANSCRIPT.

### PERFORMANCE
Style: Warm, thoughtful, conversational, emotionally alive.
Pace: Natural podcast pace with small pauses after important ideas.
Accent: Natural Russian speech.

#### TRANSCRIPT
[thoughtfully] Сегодня разберём Gemini TTS, [short pause] и почему хороший голос начинается с режиссуры.

Use English service directions and English audio tags; keep the transcript in the target language. Use PERFORMANCE for global emotion and inline tags such as [thoughtfully], [sighs], [laughs], [gasp], [medium pause] for local delivery changes. For long podcasts, prefer semantic chunks of roughly 250-450 Russian words, or 180-350 words for emotional material.

Do not paste the whole prompt skeleton into the script body. In this project, put direction into style_prompt, vibe, and speaker profile; keep body text as spoken transcript only. See the full guides: docs/skills/voiceover-pipeline/docs/11-gemini-prompting.md and docs/skills/voiceover-pipeline/docs/12-gemini-prompting-templates.md.

Результат (JSON stdout):

{"status": "success", "provider": "polza-chat-audio", "run_id": "prod",
 "duration_ms": 25520, "segment_count": 8, "cost": {"total": 0.0146, "currency": "RUB"}}

Что на выходе

out/<run-id>/
├── manifest.json                          ← entry-point для агентов
├── run_state.json                         ← resumable state after each chunk
├── generation.log                         ← human-readable generation log
├── <run-id>-voiceover-<model>.mp3         ← полный MP3
├── <run-id>-voiceover-<model>.json        ← run-манифест
├── <run-id>.timings.json                  ← Whisper-сегменты (ms)
├── <run-id>.srt                           ← субтитры SRT
└── chunks/
    ├── chunk_01.mp3 ... chunk_NN.mp3
    └── chunks.json

Команды

Все команды поддерживают --json; generate также поддерживает --json-events.

Модели и цены

Тестирование

uv sync --group dev --extra timing-whisper
uv run pytest

120 тестов: JSON-контракт, exit codes, валидация, output policy, providers, prompt modes, script metadata, resume/retry/state/log/status/concat safety, Gemini safe tags.

Agent Skill

OpenCode agent skill — устанавливает pipeline, выбирает провайдера, генерирует озвучку:

• Source: https://github.com/VladimirMonin/voiceover-pipeline/tree/v0.4.5/docs/skills/voiceover-pipeline
• Download: https://github.com/VladimirMonin/voiceover-pipeline/releases/tag/v0.4.5

Legal / Provider Notes

This project is an independent CLI wrapper around third-party providers. It is not affiliated with OpenAI, Google, OpenRouter, Polza.ai, Qwen, or CTranslate2 / faster-whisper. Provider names and model names are used solely for integration and documentation purposes. Generated audio usage is subject to the selected provider's terms of service. Do not upload private voice samples or generated speech without permission.

License

MIT. See LICENSE.

Документация

• Agent Development Workflow — безопасная работа с dirty tree, Kanban и Git/release approvals
• Agent CLI Contract — JSON-контракт, exit codes, stdout/stderr, safety rules
• Remotion Workflow — как агент Remotion использует pipeline
• Artifacts & Analysis — JSON-схемы, обработка аудио, сравнение моделей
• Whisper Timing — модель, device, compute, word timestamps
• Troubleshooting — типовые ошибки и их коды
• Polza Models — голоса, цены, особенности
• Polza TTS Models — OpenAI TTS, ElevenLabs через Polza AI
• OpenRouter TTS — Gemini, OpenAI TTS через OpenRouter
• OpenCode Skill — Agent skill для установки и озвучки
• Gemini Prompting Skill Guide — режиссура Gemini TTS, эмоции, теги, chunk limits
• Gemini Prompting Templates — project-native examples, шаблоны, QA checklist
• Qwen Local — preset-голоса, клонирование, GPU