ttsforge

Convert EPUB files to audiobooks using Kokoro ONNX TTS.

ttsforge is a command-line tool that transforms EPUB ebooks into high-quality audiobooks with support for 54 neural voices across 9 languages.

Features

• EPUB to Audiobook: Convert EPUB files to M4B, MP3, WAV, FLAC, or OPUS
• 54 Neural Voices: High-quality TTS in 9 languages
• SSMD Editing: Edit intermediate SSMD files to fine-tune pronunciation and pacing
• Custom Phoneme Dictionary: Control pronunciation of names and technical terms
• Auto Name Extraction: Automatically extract names from books for phoneme customization
• Mixed-Language Support: Auto-detect and handle multiple languages in text
• Resumable Conversions: Interrupt and resume long audiobook conversions
• Phoneme Pre-tokenization: Pre-process text for faster batch conversions
• Configurable Filenames: Template-based output naming with book metadata
• Voice Blending: Mix multiple voices for custom narration
• ONNX Runtime Providers: CPU, CUDA, NNAPI, XNNPACK, and other PyKokoro providers
• Chapter Support: M4B files include chapter markers from EPUB
• EPUB Structure Preservation: Markdown extraction keeps chapter headings, paragraphs, scene breaks, and inline emphasis in generated SSMD
• Streaming Read: Listen to EPUB/text directly with the read command
• Bounded Paragraph Rendering: Retain ordered paragraph/title render units for low-memory resume

Installation

pip install ttsforge

The base installation includes the CPU ONNX Runtime provider. Provider-dependent modules are loaded only when rendering audio, so import ttsforge and ttsforge --help remain usable for configuration and inspection operations.

Optional extras:

# Audio playback (required for --play and read)
pip install "ttsforge[audio]"

# Bundled ffmpeg (if you cannot install system ffmpeg)
pip install "ttsforge[static_ffmpeg]"

# GPU acceleration (CUDA; use a fresh environment when replacing the CPU provider)
pip install "ttsforge[gpu]"

TTSForge uses pykokoro[cpu]>=0.8.4,<0.9 and kokorog2p[espeak,en]>=0.8.0,<0.9. The supported stack prepares ordinary written forms such as dates, times, measurements, currency, ordinals, and abbreviations for speech in kokorog2p before G2P. TTSForge does not duplicate that normalization, and explicit SSMD say-as remains an author-controlled override.

It uses compact segment results and releases completed chapter audio before the next chapter starts. Chapter conversion remains chapter-buffered. Paragraph conversion prepares each chapter once, renders bounded units sequentially, and retains each unit immediately for low-memory resume.

To log opt-in process-memory snapshots during conversion:

TTSFORGE_MEMORY_DEBUG=1 ttsforge convert book.epub

Diagnostics report RSS, peak RSS, available memory, and the effective ONNX provider at runner, chapter, merge, and cleanup phases. A stable high RSS after release can reflect native allocator high-water behavior and does not by itself prove a provider leak.

Dependencies

• ffmpeg: Required for MP3/FLAC/OPUS/M4B output and chapter merging
• espeak-ng: Required for phonemization
• spaCy (optional): Automatic mode uses the highest compatible installed local model and falls back when none is available; strict requests require a model
• sounddevice (optional): Required for audio playback (--play, read)

Ubuntu/Debian:

sudo apt-get install ffmpeg espeak-ng

macOS:

brew install ffmpeg espeak-ng

spaCy models (optional):

pip install spacy
python -m spacy download en_core_web_sm
python -m spacy download en_core_web_md

Quick Start

# Convert an EPUB to audiobook (M4B with chapters)
ttsforge convert book.epub

# Use a specific voice
ttsforge convert book.epub -v am_adam

# Convert specific chapters
ttsforge convert book.epub --chapters 1-5

# List available voices
ttsforge voices

# Generate a voice demo
ttsforge demo

# Read an EPUB aloud (streaming playback)
ttsforge read book.epub

Usage

Paragraph-wise conversion

Chapter output remains the default. Use --conversion-unit paragraph for paragraph-level resume and one retained WAV per render unit:

ttsforge convert book.epub --conversion-unit paragraph
ttsforge convert book.epub --conversion-unit paragraph --yes
ttsforge convert book.epub --fresh --conversion-unit paragraph

--split-mode paragraph controls internal splitting and batching; it does not choose output files or resume granularity. The conversion unit is saved when the workspace is created, restored on resume, and cannot be changed without --fresh. --fresh is a workspace lifecycle command, not a generation setting.

Paragraph WAVs remain in <output-stem>_paragraphs/ with manifest.json, marker sidecars, and playlist.m3u8. A render unit is an optional announced chapter-title unit followed by spoken paragraph units. Fixed-width global sequence prefixes make lexical order playback order. PyKokoro owns paragraph and boundary pauses; TTSForge-owned interchapter silence is stored only in the final unit of each non-final selected chapter. A complete workspace can be merged without initializing ONNX. See examples/README.md and the conversion, resume, and manifest examples.

Paragraph mode saves progress after every finalized unit. To start and resume a paragraph conversion, use:

ttsforge convert "Platform Decay - Martha Wells.epub" --fresh --conversion-unit paragraph
# Interrupt the conversion, then run the normal command:
ttsforge convert "Platform Decay - Martha Wells.epub"

The second command restores the saved paragraph mode, chapter selection, output path, and omitted audio-affecting settings. When --seed is omitted, TTSForge creates and persists a hidden non-negative preparation seed for each chapter before stochastic short-sentence processing begins, then reuses it across processes. Fresh conversions receive new automatic seeds; --seed 42 supplies the same explicit seed to every chapter. An explicitly changed audio-affecting option rejects resume and reports the changed field; remove the override to use the saved value or use --fresh to deliberately start a new workspace. Verifiable schema-6 workspaces migrate to schema 7; unverifiable legacy workspaces and their artifacts are preserved. Paragraph workspaces created with older than schema 6 cannot guarantee deterministic preparation and require a fresh run.

Basic Conversion

ttsforge convert book.epub

Creates book.m4b with default settings (voice: af_heart, format: M4B).

Voice Selection

# List all voices
ttsforge voices

# List voices for a language
ttsforge voices -l b  # British English

# Convert with specific voice
ttsforge convert book.epub -v bf_emma

Output Formats

ttsforge convert book.epub -f mp3    # MP3
ttsforge convert book.epub -f wav    # WAV (uncompressed)
ttsforge convert book.epub -f flac   # FLAC (lossless)
ttsforge convert book.epub -f opus   # OPUS
ttsforge convert book.epub -f m4b    # M4B audiobook (default)

Chapter Selection

# Preview chapters
ttsforge list book.epub

# Convert range
ttsforge convert book.epub --chapters 1-5

# Convert specific chapters
ttsforge convert book.epub --chapters 1,3,5,7

# Mixed selection
ttsforge convert book.epub --chapters 1-3,5,10-15

Speed Control

ttsforge convert book.epub -s 1.2   # 20% faster
ttsforge convert book.epub -s 0.9   # 10% slower

Resumable Conversions

Conversions are resumable by default. If interrupted, re-run the same command:

ttsforge convert book.epub  # Resumes from last chapter
ttsforge convert book.epub --fresh  # Start over

For paragraph conversion, the resume summary reports completed units and the next chapter/paragraph. Valid WAV and marker artifacts are retained, while a damaged unit invalidates only its ordered suffix.

Phoneme Workflow

For large books or batch processing, pre-tokenize to phonemes:

# Export to phonemes (fast, CPU-only)
ttsforge phonemes export book.epub

# Convert phonemes to audio (can run on different machine)
ttsforge phonemes convert book.phonemes.json -v am_adam

Configuration

# View settings
ttsforge config --show

# Set defaults
ttsforge config --set default_voice am_adam
ttsforge config --set default_format mp3
ttsforge config --set onnx_provider nnapi

# Reset to defaults
ttsforge config --reset

The legacy two-token configuration option remains available and may be repeated. Values beginning with a dash are accepted as values rather than being interpreted as options:

ttsforge config --set default_speed 1.1 --set default_title -draft

Advanced short-sentence handling can be managed with a JSON config:

# Create and link the advanced short-sentence config
ttsforge config short-sentence init

# Show the advanced short-sentence config
ttsforge config short-sentence show

# Reset the advanced short-sentence config to defaults
ttsforge config short-sentence reset

The former short-sentence-advanced-config root command remains available as a deprecated compatibility alias.

Filename Templates

Customize output filenames with metadata:

ttsforge config --set output_filename_template "{author} - {book_title}"

Available variables: {book_title}, {author}, {chapter_title}, {chapter_num}, {input_stem}, {chapters_range}

Voices

ttsforge includes 54 voices across 9 languages:

Language	Code	Voices	Default
American English	a	20	af_heart
British English	b	8	bf_emma
Spanish	e	3	ef_dora
French	f	1	ff_siwis
Hindi	h	4	hf_alpha
Italian	i	2	if_sara
Japanese	j	5	jf_alpha
Brazilian Portuguese	p	3	pf_dora
Mandarin Chinese	z	8	zf_xiaoxiao

Voice naming: {lang}{gender}_{name} (e.g., am_adam = American Male "Adam")

Voice Demo

# Demo all voices
ttsforge demo

# Demo specific language
ttsforge demo -l a

# Save individual voice files
ttsforge demo --separate -o ./voices/

Voice Blending

Mix multiple voices for custom narration:

# Using --voice parameter (auto-detects blend format)
ttsforge convert book.epub --voice "af_nicole:50,am_michael:50"

# Using --voice-blend parameter (traditional method)
ttsforge convert book.epub --voice-blend "af_nicole:50,am_michael:50"

# Weighted blends (70% Nicole, 30% Michael)
ttsforge convert book.epub --voice "af_nicole:70,am_michael:30"

# Works with all commands
ttsforge sample "Hello world" --voice "af_sky:60,bf_emma:40" -p
ttsforge phonemes preview "Test blend" --voice "am_adam:50,am_michael:50" --play

Mixed-Language Support

For books with multiple languages (e.g., German text with English technical terms):

# Enable mixed-language auto-detection
ttsforge convert book.epub \
  --use-mixed-language \
  --mixed-language-primary de \
  --mixed-language-allowed de,en-us

# Test with a sample
ttsforge sample \
  "Das ist ein deutscher Satz. This is an English sentence." \
  --use-mixed-language \
  --mixed-language-primary de \
  --mixed-language-allowed de,en-us

Requirements: Install lingua-language-detector for automatic language detection:

pip install lingua-language-detector

Configuration options:

• --use-mixed-language - Enable mixed-language mode
• --mixed-language-primary LANG - Primary language (e.g., de, en-us)
• --mixed-language-allowed LANGS - Comma-separated list of allowed languages
• --mixed-language-confidence FLOAT - Detection confidence threshold (0.0-1.0, default: 0.7)

Supported languages: en-us, en-gb, de, fr-fr, es, it, pt, pl, tr, ru, ko, ja, zh/cmn

SSMD Editing

ttsforge uses SSMD (Speech Synthesis Markdown) as an intermediate format between your EPUB and the final audio. This allows you to fine-tune pronunciation, pacing, and emphasis before conversion.

How It Works

During conversion, ttsforge automatically generates .ssmd files for each chapter:

.{book_title}_chapters/
├── chapter_001_intro.ssmd      # Editable text with speech markup
├── chapter_001_intro.wav
├── chapter_002_chapter1.ssmd
├── chapter_002_chapter1.wav

When you resume a conversion, ttsforge detects if you've edited any SSMD files and automatically regenerates the audio.

Basic Workflow

# 1. Start conversion
ttsforge convert book.epub

# 2. Pause conversion (Ctrl+C)

# 3. Edit SSMD files to fix pronunciation or pacing
vim .book_chapters/chapter_001_intro.ssmd

# 4. Resume - automatically detects edits and regenerates audio
ttsforge convert book.epub

SSMD Syntax

SSMD files use a simple markdown-like syntax:

Structural Breaks (control pauses):

...p    # Paragraph break (0.5-1.0s pause)
...s    # Sentence break (0.1-0.3s pause)
...c    # Clause break (shorter pause)

Emphasis:

*text*      # Moderate emphasis
**text**    # Strong emphasis

EPUB processing has three independent layers:

1. Semantic extraction: epub2text converts EPUB navigation, headings, paragraphs, scene breaks, semantic emphasis, and CSS emphasis into chapter Markdown.
2. SSMD generation: TTSForge preserves that controlled Markdown in .ssmd files; ## headings and */** spans remain editable and parseable.
3. Audible rendering: --emphasis-level controls gain-only audible strength while --ssmd-emphasis remains available for advanced policy handling.

Markdown extraction and emphasis preservation are enabled by default, while audible emphasis remains plain:

# Default: Markdown structure and emphasis are preserved; audio remains plain
ttsforge convert book.epub

# Unwrap emphasis while retaining headings and scene breaks
ttsforge convert book.epub --no-detect-emphasis

# Compare with the legacy flattened extraction path
ttsforge convert book.epub --epub-content-mode plain

# Light, normal, or strong gain-only audible emphasis
ttsforge convert book.epub --emphasis-level 1
ttsforge convert book.epub --emphasis-level 2
ttsforge convert book.epub --emphasis-level 3

# Persist the normal level for future conversions
ttsforge config --set emphasis_level 2
ttsforge convert book.epub

# Choose the persisted/default policy explicitly
ttsforge config --set ssmd_emphasis_mode plain
ttsforge convert book.epub --ssmd-emphasis approximate

# Select AudioSig prosody for explicit SSMD rate/pitch annotations
ttsforge config --set prosody_method esola
ttsforge convert book.epub --prosody-method psola

epub_content_mode, detect_emphasis, emphasis_level, ssmd_emphasis_mode, and prosody_method are separate settings. The first selects Markdown or explicit legacy plain extraction; the second preserves or unwraps inline emphasis without affecting headings; the third controls friendly audible strength; the fourth remains the advanced SSMD policy; and the fifth selects AudioSig processing for explicit rate and pitch annotations. The default preserves EPUB emphasis but leaves automatic audible emphasis off. Level 2 is the backward-compatible equivalent of --enable-ssmd-emphasis; that legacy flag remains available with a deprecation warning. psola is accepted as an alias for AudioSig's canonical td_psola.

The user-friendly levels are 0=Off, 1=Light, 2=Normal, and 3=Strong. The advanced policies are plain, approximate, warn, and error. Explicit SSMD prosody such as [fast words]{rate="fast"} remains active in plain emphasis mode.

spaCy model policy

TTSForge uses the released PyKokoro/phrasplit model policy for sentence segmentation and G2P. When no exact model or tier is configured, it selects the highest installed compatible local model for each effective language; model packages are never downloaded by this selection, and conversion falls back without a local model. --spacy (or use_spacy=true), an exact package, and an exact tier are strict; --no-spacy is disabled mode. The request is visible in the conversion summary and the concrete selection is frozen into conversion state and resume identity.

# Quality-first automatic behavior (omit both keys)
ttsforge convert book.epub

# Preserve a previous medium-tier workflow
ttsforge config --set spacy_model_size md

# Preserve a previous small-model workflow exactly
ttsforge convert book.epub --spacy-model en_core_web_sm

# Disable spaCy in the conversion pipeline
ttsforge convert book.epub --no-spacy

Use --spacy-model for a strict package request or --spacy-model-size for a strict sm, md, lg, or trf tier request. Explicit packages take precedence over tiers. Changing the installed model set can change segmentation, phonemes, and audio; old resumable conversions may therefore require --fresh.

Custom Phonemes:

[Hermione]{ph="hɝmˈIni"}    # Override pronunciation
[API]{ph="ˌeɪpiˈaɪ"}        # Technical terms

Language annotations (supported):

[Bonjour]{lang="fr"}    # Mark text as French

Example SSMD File

Chapter One ...p

[Harry]{ph="hæɹi"} Potter was a *highly unusual* boy in many ways. ...s
For one thing, he **hated** the summer holidays more than any other
time of year. ...s For another, he really wanted to do his homework,
but was forced to do it in secret, in the dead of the night. ...p

And he also happened to be a wizard. ...p

When to Use SSMD Editing

• Pronunciation issues: Character names, technical terms, foreign words
• Pacing problems: Adjust paragraph and sentence breaks
• Emphasis corrections: Add or remove emphasis on specific words
• Combine with phoneme dictionary: Phoneme dictionary applied automatically to SSMD

For detailed SSMD 0.8 syntax, validation, policy options, and Kokoro limitations, see docs/ssmd.md.

Custom Phoneme Dictionary

Control pronunciation of character names, technical terms, and foreign words with custom phoneme dictionaries.

Quick Start

# 1. Extract names from your book (requires spacy)
ttsforge extract-names mybook.epub

# 2. Review the generated custom_phonemes.json file
ttsforge list-names custom_phonemes.json

# 3. Test pronunciation with sample
ttsforge sample "Hermione loves Kubernetes" --phoneme-dict custom_phonemes.json -p

# 4. Convert with custom pronunciations
ttsforge convert mybook.epub --phoneme-dict custom_phonemes.json

Requirements

For automatic name extraction (optional but recommended):

pip install spacy
# Install the compatible package(s) you want to make available locally.
python -m spacy download en_core_web_lg

Name extraction accepts --spacy-model, --spacy-model-size, and --language; it selects the highest installed model with the required NER capability and records the concrete package in dictionary metadata.

Workflow

1. Extract names from your book:

# Extract frequent names (≥3 occurrences)
ttsforge extract-names mybook.epub

# Preview without saving
ttsforge extract-names mybook.epub --preview

# Only very frequent names (≥10 occurrences)
ttsforge extract-names mybook.epub --min-count 10 -o names.json

# Include all proper nouns, not just detected person names
ttsforge extract-names mybook.epub --include-all

This creates a custom_phonemes.json file with auto-generated phoneme suggestions.

2. Review and edit the dictionary:

# List all entries
ttsforge list-names custom_phonemes.json

# Sort alphabetically
ttsforge list-names custom_phonemes.json --sort-by alpha

Edit custom_phonemes.json to fix any incorrect phonemes. The file format is:

{
  "_metadata": {
    "generated_from": "mybook.epub",
    "language": "en-us"
  },
  "entries": {
    "Hermione": {
      "phoneme": "hɝmˈIni",
      "occurrences": 847,
      "verified": false
    },
    "Kubernetes": {
      "phoneme": "kubɚnˈɛtɪs",
      "occurrences": 12,
      "verified": false
    }
  }
}

Or use the simple format:

{
  "Hermione": "hɝmˈIni",
  "Kubernetes": "kubɚnˈɛtɪs"
}

3. Test pronunciation:

# Test specific names
ttsforge sample "Hermione and Harry" --phoneme-dict custom_phonemes.json -p

# Test and save to file
ttsforge sample "Hermione and Harry" --phoneme-dict custom_phonemes.json -o test.wav

4. Convert your book:

# Use the dictionary for conversion
ttsforge convert mybook.epub --phoneme-dict custom_phonemes.json

# Case-sensitive matching (default is case-insensitive)
ttsforge convert mybook.epub \
  --phoneme-dict custom_phonemes.json \
  --phoneme-dict-case-sensitive

Manual Dictionary Creation

You can create a dictionary manually without extraction:

{
  "Katniss": "kætnɪs",
  "Peeta": "pitə",
  "Panem": "pænəm"
}

Getting IPA Phonemes

To find the correct IPA phonemes for a word:

1. Use ttsforge sample "word" -p to hear the default pronunciation
2. Look up IPA pronunciation online (e.g., Wiktionary, IPA dictionaries)
3. Or use the auto-generated phonemes as a starting point

Note: Phoneme matching is case-insensitive by default and respects word boundaries (e.g., "test" won't match "testing").

Commands

The CLI is built from explicit typed Typer command wrappers. Help and version paths do not initialize the ONNX provider; only executing a backend-dependent command imports its implementation module.

Command	Description
convert	Convert EPUB to audiobook
list	List chapters in EPUB
info	Show EPUB metadata
sample	Generate sample audio
read	Stream playback from EPUB/text
voices	List available voices
demo	Generate voice demo
extract-names	Extract names for phoneme dictionary
list-names	List names in phoneme dictionary
download	Download ONNX models
config	Manage configuration
phonemes export	Export EPUB to phonemes
phonemes convert	Convert phonemes to audio
phonemes info	Show phoneme file info
phonemes preview	Preview text as phonemes

ONNX Runtime Providers

Select an ONNX Runtime execution provider globally or per command. The value may be an alias (auto, cpu, cuda, openvino, directml/dml, coreml, nnapi, or xnnpack) or a full *ExecutionProvider name:

ttsforge config --set onnx_provider nnapi
ttsforge sample "NNAPI test" --provider nnapi
ttsforge sample "CPU test" --provider CPUExecutionProvider

The legacy Boolean flags remain compatibility shortcuts: --gpu maps to auto and --no-gpu maps to cpu. NNAPI and XNNPACK are execution providers, not GPU modes. PyKokoro applies its documented ONNX_PROVIDER environment override after TTSForge resolves configuration.

ttsforge convert book.epub --gpu
ttsforge convert book.epub --provider xnnpack

For Termux/Android, install an ONNX Runtime build compatible with the declared PyKokoro release, then configure the GitHub assets explicitly:

ttsforge config \
  --set model_source github \
  --set model_variant v1.0 \
  --set model_quality fp32 \
  --set onnx_provider nnapi
ttsforge config --show
ttsforge download
ttsforge sample "Termux provider test" --provider nnapi

config --show reports the configured source/variant/quality. If that set is incomplete but the other supported source has a complete set, it reports the alternate without switching sources automatically. With the required patched PyKokoro release, GitHub v1.0 uses the embedded standard vocabulary and does not download Hugging Face config.json. Provider availability depends on the installed Android ONNX Runtime build; use another available provider if NNAPI is not exposed.

Configuration Options

Option	Default	Description
default_voice	af_heart	Default TTS voice
default_language	a	Default language code
default_speed	1.0	Speech speed (0.5-2.0)
default_format	m4b	Output format
onnx_provider	cpu	ONNX Runtime provider alias or full name
use_gpu	false	Legacy shortcut (true => auto)
model_quality	fp32	Model quality/quantization
model_variant	v1.0	Model variant
silence_between_chapters	2.0	Chapter gap (seconds)
pause_clause	0.5	Clause pause (seconds)
pause_sentence	0.7	Sentence pause (seconds)
pause_paragraph	0.9	Paragraph pause (seconds)
pause_variance	0.05	Pause variance (seconds)
pause_mode	auto	Pause mode (tts, manual, auto)
enable_short_sentence	None	Handle short sentences
announce_chapters	true	Speak chapter titles
chapter_pause_after_title	2.0	Pause after chapter title
phonemization_lang	None	Override phonemization language
output_filename_template	{book_title}	Output filename template
default_content_mode	chapters	read mode (chapters/pages)
default_page_size	2000	Page size for read pages mode
use_mixed_language	false	Enable mixed-language mode
mixed_language_primary	None	Primary language for mixed mode
mixed_language_allowed	None	Allowed languages (list)
mixed_language_confidence	0.7	Language detection threshold

Documentation

Full documentation: https://ttsforge.readthedocs.io/

Build locally:

pip install -r docs/requirements.txt
make html

Requirements

• Python 3.10+
• ffmpeg (for MP3/FLAC/OPUS/M4B output and chapter merging)
• espeak-ng (for phonemization)
• ~330MB disk space (ONNX models)
• sounddevice (optional, for audio playback)

License

MIT License

Credits

• Kokoro - TTS model
• espeak-ng - Phonemization
• ONNX Runtime - Model inference