whisperspeech2 0.9.1

An Open Source text-to-speech system built by inverting Whisper (fork of WhisperSpeech)

WhisperSpeech2

An Open Source text-to-speech system built by inverting Whisper. This is a fork of WhisperSpeech optimized for inference. The creators of the original project abandoned it, hence this fork.

Installation

pip install whisperspeech2

Note: You must also have PyTorch installed. Visit pytorch.org for installation instructions.

Quick Start

from whisperspeech2.pipeline import Pipeline

# Initialize the pipeline
pipe = Pipeline(
    s2a_ref='WhisperSpeech/WhisperSpeech:s2a-q4-tiny-en+pl.model',
    t2s_ref='WhisperSpeech/WhisperSpeech:t2s-tiny-en+pl.model'
)

# Generate audio and save to file
pipe.generate_to_file('output.wav', "Hello, world!")

# Or get the audio tensor directly
audio = pipe.generate("Hello, world!")

Available Models

For more details about each model, visit the WhisperSpeech Hugging Face repository.

S2A Models (Semantic to Acoustic)

Model	Reference
Tiny (Q4)	WhisperSpeech/WhisperSpeech:s2a-q4-tiny-en+pl.model
Base (Q4)	WhisperSpeech/WhisperSpeech:s2a-q4-base-en+pl.model
Small (Q4)	WhisperSpeech/WhisperSpeech:s2a-q4-small-en+pl.model
HQ Fast (Q4)	WhisperSpeech/WhisperSpeech:s2a-q4-hq-fast-en+pl.model
v1.1 Small	WhisperSpeech/WhisperSpeech:s2a-v1.1-small-en+pl.model
v1.95 Small Fast	WhisperSpeech/WhisperSpeech:s2a-v1.95-small-fast-en.model

T2S Models (Text to Semantic)

Model	Reference
Tiny	WhisperSpeech/WhisperSpeech:t2s-tiny-en+pl.model
Base	WhisperSpeech/WhisperSpeech:t2s-base-en+pl.model
Small	WhisperSpeech/WhisperSpeech:t2s-small-en+pl.model
Fast Small	WhisperSpeech/WhisperSpeech:t2s-fast-small-en+pl.model
Fast Medium	WhisperSpeech/WhisperSpeech:t2s-fast-medium-en+pl+yt.model
HQ Fast	WhisperSpeech/WhisperSpeech:t2s-hq-fast-en+pl.model
v1.1 Small	WhisperSpeech/WhisperSpeech:t2s-v1.1-small-en+pl.model

Model Recommendations

Use Case	S2A Model	T2S Model	VRAM	Speed
Lowest Resources	s2a-q4-tiny	t2s-tiny	~450 MB	~16s
Best Speed	s2a-v1.95-small-fast	t2s-tiny	~1.7 GB	~15s
Balanced	s2a-q4-hq-fast	t2s-tiny	~1.7 GB	~15s
Higher Quality	s2a-q4-hq-fast	t2s-hq-fast	~2.1 GB	~16s

Avoid: Combinations using s2a-q4-small or s2a-v1.1-small with t2s-fast-medium result in high VRAM (~4GB) and slow processing (~42s).

Speaker Embedding (Optional)

To use custom speaker embeddings, install the optional dependency:

pip install whisperspeech2[speaker]

Then pass an audio file path to clone a voice:

pipe.generate_to_file('output.wav', "Hello!", speaker='reference.wav')

Examples

See the examples/ directory for more usage examples including GUI applications and streaming playback.

License

MIT License