spantrack

Convert a text EPUB into a self-contained EPUB 3 that narrates itself and highlights each sentence as it reads — an open, fully offline take on Kindle's Immersion Reading.

What it does

Synchronized read-along narration, text that highlights in time with spoken audio, is effectively locked to Amazon's ecosystem and its paired audiobook purchases. spantrack builds the same thing from any DRM-free text EPUB, on your own machine: it voices the book with a local neural TTS model, wires the audio into standard EPUB 3 Media Overlays, and hands back a single *.narrated.epub that plays with correct highlighting in any Media-Overlay reader — Thorium on desktop, BookFusion on mobile. No account, no cloud, no network after a one-time model download.

Demo

https://github.com/user-attachments/assets/a1d52881-4d21-45b1-997e-f201dc9d56f8

Watch the demo — a spantrack read-along playing in Thorium.

How it works

The hard part of a read-along is keeping the highlight glued to the audio. Most tools synthesize or record a whole chapter and then run a forced aligner to guess where each sentence begins — which drifts (seconds of lag in comparable prior art). spantrack never aligns anything.

[!NOTE] Sync is exact by construction. Each sentence is synthesized on its own, the chapter audio is those sentences concatenated in order, and every SMIL clip offset is the running sum of real clip durations. The timestamp is the audio position, so there is nothing to drift.

The pipeline is one stage per module:

1. Ingest — unzip a working copy, parse the OPF, and upgrade EPUB 2 to EPUB 3 (synthesizing nav.xhtml from the NCX) when needed.
2. Select — walk the spine, skip cover / nav / front-matter by epub:type landmarks, and print the choice; --include / --exclude / --pick override it.
3. Segment & tag — walk block elements, split each into sentences with pysbd, and wrap every sentence in <span id="sNNNNN">, preserving all inline markup.
4. Normalize (TTS input only — the DOM is never touched) — expand numbers, dates, currency, roman numerals, footnote markers, and URLs; apply an optional per-book pronunciation lexicon.
5. Synthesize — one sentence at a time through Kokoro-82M at a fixed voice, so the narrator stays consistent across the whole book. Output is content-addressed by sha256(model · voice · speed · text), so a killed run resumes without re-synthesizing and identical text is never voiced twice.
6. Assemble — concatenate sentences with natural gaps and short edge fades, then encode one mono MP3 (or Opus) per chapter with ffmpeg.
7. Overlay — emit one gapless SMIL per chapter, patch the OPF (media-overlay wiring, media:duration / narrator), and inject highlight CSS.
8. Repack & validate — zip mimetype-first, then optionally gate the build on epubcheck.

Publisher CSS and markup stay byte-for-byte identical outside the injected spans, so turning narration off leaves the book looking exactly like the source.

Install

pip install spantrack

spantrack's grapheme-to-phoneme step also needs spaCy's en_core_web_sm, which isn't published to PyPI — install the pinned wheel explicitly. Pinning keeps G2P offline and deterministic; otherwise misaki downloads a model at runtime.

pip install "en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl"

You also need:

• ffmpeg on your PATH, to encode chapter audio. Not required for --dry-run or --oov-report.
• A JRE, only for --validate, which runs epubcheck on the output.

The first real conversion downloads the Kokoro weights once (a few hundred MB); after that it runs fully offline. CUDA is used automatically when present, otherwise synthesis falls back to CPU.

Usage

spantrack book.epub

This writes book.narrated.epub beside the input and a book.epub.spantrack/ work directory holding the sentence cache and per-chapter checkpoints. Re-running resumes from there; --clean discards it.

Handy flags:

• --dry-run — print the narration set and a duration/size estimate, synthesizing nothing.
• --oov-report — list words the voice may mispronounce, as a lexicon skeleton to fill in.
• --voice af_heart / --speed 1.1 / --lang en-us|en-gb — narrator, pace, and accent.
• --lexicon words.toml — per-book pronunciation overrides in misaki IPA.
• --include / --exclude / --pick — steer which documents get narrated.
• --audio-format opus / --bitrate 64k — codec and size.
• --granularity word — experimental word-by-word highlighting (timings drift; A/B a single chapter first).
• --validate — run epubcheck and fail the build on errors.
• --progress — a per-chapter progress bar (needs pip install 'spantrack[progress]').

Run spantrack --help for the complete list.

Examples

The examples/ directory ships four hand-written sample books, plus short, commented Python scripts that drive the API. Convert one end to end:

spantrack examples/the-lighthouse-keeper.epub --validate

or just inspect the plan without synthesizing:

spantrack examples/the-pocket-anthology.epub --dry-run

examples/README.md walks through each book and script with captured output.

Status

[!WARNING] Beta. English only, one fixed-voice narrator, and word-mode highlighting is experimental. The output validates with epubcheck and plays correctly in Thorium; the mobile (BookFusion) target is not yet validated first-hand. Interfaces may change without notice.

License

MIT.