ZeroTTS

Vietnamese Zero-Shot Text-to-Speech (TTS) with real-time streaming and voice cloning from seconds of audio. Fast, natural, and optimised for CPU inference.

The most accurate open Vietnamese TTS we know of — 4× fewer word errors than the next open model, and it runs faster than real time on a laptop CPU.

• 🗣️ Zero-shot voice cloning — cloned from as little as 3 seconds of reference audio (up to 30 seconds). No fine-tuning, no per-speaker training.
• ⚡ Real-time on CPU, streaming — ~2× faster than real time (RTF 0.5×), first audio chunk in ~70 ms. No GPU required.
• 🇻🇳 Built for Vietnamese — tones, code-switched English, and reads words like 31/12/2026 and ZeroTTS without text normalizer.

Samples

Contents

1. Samples
2. Install
3. Usage
4. Voices — and voice cloning
5. Benchmarks
6. Web UI
7. Browser demo
8. How it works
9. Credits

Install

pip install zerotts

That pulls numpy, onnxruntime, tokenizers, huggingface_hub, soundfile, scipy — and nothing else. Weights download from the Hub on first use (~900 MB, cached under HF_HOME).

Optional extras:

pip install "zerotts[webui]"   # Gradio demo
pip install "zerotts[eval]"    # benchmark scorers — these DO need torch

The eval extra is the only thing in this repo that installs PyTorch, and it is for measuring quality, not for generating audio.

Usage

Web UI

pip install "zerotts[webui]"
python webui/app.py                       # http://localhost:7860
python webui/app.py --model ./local_dir   # a local model directory

Voice picker, streaming playback, long-form segmentation, and the generation settings above.

Browser demo

js/ runs the same model client-side with onnxruntime-web — no server, no upload. See docs/BROWSER.md.

Note the download: the weights are fp32 and not quantized, so the demo fetches ~900 MB once and persists it (OPFS/Cache API). That is a deliberate quality-over-size choice; it targets desktop broadband, not mobile data.

Python

from zerotts import ZeroTTS

tts = ZeroTTS.from_pretrained("zeroweight-ai/ZeroTTS")

print(tts.list_voices())

# One shot
audio = tts.synthesize("Hôm nay trời đẹp quá.", voice="maichi")
tts.save_audio(audio, "out.wav")

# Streaming — first chunk arrives in ~70 ms
import queue

import numpy as np
import sounddevice as sd   # pip install sounddevice

TEXT = ("Đây là chế độ phát trực tuyến. Âm thanh được tạo ra và phát ngay lập tức, "
        "không cần chờ toàn bộ đoạn văn hoàn thành. Nhờ vậy, người nghe chỉ mất "
        "khoảng 70 mili giây là đã nghe thấy câu đầu tiên, ngay cả khi mô "
        "hình đang chạy trên CPU của một chiếc laptop bình thường.")

pending, tail = queue.Queue(), np.zeros(0, dtype="float32")

def feed(outdata, frames, _time, _status):
    global tail
    while len(tail) < frames and not pending.empty():
        tail = np.concatenate([tail, pending.get_nowait()])
    n = min(frames, len(tail))
    outdata[:n, 0] = tail[:n]
    outdata[n:] = 0
    tail = tail[n:]

with sd.OutputStream(samplerate=tts.sample_rate, channels=1,
                     dtype="float32", callback=feed):
    for chunk in tts.synthesize_stream(TEXT, voice="maichi"):
        pending.put(chunk.reshape(-1))   # chunk is (1, n) float32 at 48 kHz
    while not pending.empty() or len(tail):
        sd.sleep(50)                     # let the buffer drain before closing

Dates, clock times, fractions and acronyms are expanded to spoken Vietnamese before synthesis:

from zerotts import normalize_vi_text

normalize_vi_text("Ngày 23/8/2024 lúc 15h30, giá 1.250.000")
# 'Ngày hai mươi ba tháng tám năm hai nghìn không trăm hai mươi tư lúc
#  mười lăm giờ ba mươi phút, giá một triệu hai trăm năm mươi nghìn'

synthesize() does not apply it — it is a separate step so you stay in control (the expansions are Vietnamese words, so they are wrong for English text). The CLI and web UI apply it by default; zerotts say --no_text_norm turns it off.

Long input should be segmented — the model is trained on utterances, not paragraphs:

from zerotts.chunking import chunk_text, clean_segment_punctuation, normalize_punctuation

segments = [clean_segment_punctuation(s)
            for s in chunk_text(normalize_punctuation(long_text), max_chunk_sec=15)]

Command line

zerotts voices
zerotts say "Xin chào các bạn." --voice maichi -o hello.wav
zerotts say "$(cat article.txt)" --voice maichi --chunk -o article.wav
zerotts bench --voice maichi

Generation settings

Defaults are the exact settings the benchmark numbers were produced with, so out-of-the-box output matches the published scores.

Voices — and voice cloning

A voice in ZeroTTS is a small array of speaker latents, shape (1, n_voice_queries, d_model). That array is the entire speaker conditioning — there is no reference transcript, no in-context audio prompt, no teacher-forced frames. It ships as a .npz inside the weights repo.

Voice cloning is not available in this release

Those latents are produced by a voice encoder that reads a reference clip, and the voice encoder is not published. This package can load voices; it cannot create them from audio. There is no flag that turns this on.

To clone your own voice, head to platform.zeroweight.ai/audio, upload a reference clip, and download the resulting latents.

The boundary is narrower than it sounds: latents obtained that way are just a .npz. The platform's voice library hands you one zip per voice, and every surface here takes that file as it came down — no unzipping, no copying:

tts.add_voices("my-voice.zip")   # once, ever
tts.synthesize("Xin chào.", voice="my-voice")

Both demos have it as the voice picker's second tab, Nhập giọng của bạn — drop the zip on it and the voice is installed, selected, and there again next time. The Gradio server can start with one: python webui/app.py --voice ~/Downloads/my-voice.zip. Details and the pack format: docs/VOICES.md.

Eight presets ship with the weights, each tagged by gender, age and register so you can pick one by ear or by filter — maichi (Mai Chi) is the default used throughout this README. Full list, tags, and preview clips: docs/VOICES.md.

tts.list_voices()                       # ['maichi', 'baotrang', ...]
v = tts.load_voice("maichi")
v.emb.shape                             # (1, 10, 768)
v.display_name, v.gender, v.tags        # 'Mai Chi', 'nữ', ['nữ', 'trẻ', 'kể chuyện', ...]

# A latent array from anywhere works directly
audio = tts.synthesize("…", voice=my_latents)

Benchmarks

Measured on ZeroBench-TTS

Every system reads raw text — dates, numbers and acronyms verbatim, exactly as they appear in the wild, with no text frontend in front of the model.

4× fewer word errors than the next-best system, and the fastest of the four on CPU. The gap is much wider in latency than in throughput: the two XTTS fine-tunes also beat real time (0.71×) but need seconds to emit their first sample, while OmniVoice is 6× slower than real time. All three are sized and tuned for a GPU, and it shows.

Full comparison tables, per-subset breakdowns, and CPU speed methodology: docs/BENCHMARKS.md

Speed — CPU

RTF (realtime factor, wall-clock synthesis time ÷ output audio duration — lower is faster; below 1× is faster than real time) and time-to-first-audio, all measured on CPU, single request, 8 inference threads pinned to a dedicated core pool (no other synthesis running concurrently). Three Vietnamese samples — short (26 chars), medium (77 chars), long (227 chars) — each run 6 times with the first 2 (cold-cache) discarded; figures below are the mean of the remaining 4.

ZeroTTS's time-to-first-audio comes from its real streaming path — first audio frame, not first full utterance. The three baselines have no working CPU streaming path, so their TTFA is the time to the complete utterance.

Credits

Speech codec: MOSS-Audio-Tokenizer-Nano by the OpenMOSS team, Apache-2.0. ZeroTTS bundles its ONNX decoder graphs in the weights repo so there is no external runtime dependency; see NOTICE and LICENSES/.

@misc{gong2026mossaudiotokenizerscalingaudiotokenizers,
  title={MOSS-Audio-Tokenizer: Scaling Audio Tokenizers for Future Audio Foundation Models},
  author={Yitian Gong and Kuangwei Chen and Zhaoye Fei and Xiaogui Yang and Ke Chen
          and Yang Wang and Kexin Huang and Mingshu Chen and Ruixiao Li
          and Qingyuan Cheng and Shimin Li and Xipeng Qiu},
  year={2026}, eprint={2602.10934}, archivePrefix={arXiv}, primaryClass={cs.SD}
}

Vietnamese text normalization adapts the expansion rules and abbreviation table of soe-vinorm (MIT), reimplemented as pure stdlib regex so the inference path keeps its no-torch, no-download guarantee. See NOTICE.

Benchmark reference audio comes from VIVOS, viVoice, phoaudiobook and Emilia. ASR scoring uses PhoWhisper (VinAI).

License

Code and weights: MIT. The bundled MOSS codec decoder is Apache-2.0. The ZeroBench-TTS dataset is CC-BY-NC-4.0 (it redistributes audio from the corpora above) — that applies to the benchmark, not to ZeroTTS.