whisperspeech2 0.9.3

An Open Source text-to-speech system built by inverting Whisper (fork of WhisperSpeech)

🎙️ WhisperSpeech2

An Open Source text-to-speech system built by inverting Whisper. This is a fork of WhisperSpeech optimized for inference that introduces ⚡"cuda graphs"⚡ for faster inference.

🚀 Installation

pip install whisperspeech2

PyTorch

CUDA libraries if using an Nvidia GPU (tested with CUDA 12.8).

✨ Available Models

You can mix and match the models for different quality and compute requirements. See the WhisperSpeech Hugging Face repository.

S2A Models (Semantic to Acoustic)

Model	Reference
Tiny	WhisperSpeech/WhisperSpeech:s2a-q4-tiny-en+pl.model
Base	WhisperSpeech/WhisperSpeech:s2a-q4-base-en+pl.model
Small	WhisperSpeech/WhisperSpeech:s2a-q4-small-en+pl.model
HQ Fast	WhisperSpeech/WhisperSpeech:s2a-q4-hq-fast-en+pl.model
v1.1 Small	WhisperSpeech/WhisperSpeech:s2a-v1.1-small-en+pl.model

T2S Models (Text to Semantic)

Model	Reference
Tiny	WhisperSpeech/WhisperSpeech:t2s-tiny-en+pl.model
Base	WhisperSpeech/WhisperSpeech:t2s-base-en+pl.model
Small	WhisperSpeech/WhisperSpeech:t2s-small-en+pl.model
Fast Small	WhisperSpeech/WhisperSpeech:t2s-fast-small-en+pl.model
Fast Medium	WhisperSpeech/WhisperSpeech:t2s-fast-medium-en+pl+yt.model
HQ Fast	WhisperSpeech/WhisperSpeech:t2s-hq-fast-en+pl.model

Benchmark (no cuda graph)

Benchmark (with cuda graph)

People with Nvidia GPUs can set the "use_cuda_graph" parameter to "true" for fastger processing and less VRAM usage.

Examples

See the examples/ directory for more usage examples including GUI applications and streaming playback.

Thanks/Shout Outs

Thanks to Jakub for the inspiration.

The fine folks at the Dia2 for forcing me to learn about cuda graph. Go check theirs out too.

BIG FUCK YOU

And finally a big "fuck you" to Microsoft for installing shit on my computer that I don't need and/or want for years.

Your pathetic VibeVoice project just got pwned!