Modern TTS

A unified, extensible, and future-proof Python toolkit for locally running state-of-the-art LLM-based Text-to-Speech (TTS) synthesis models.

✨ Features

• 🧩 25+ Models — MeloTTS, ChatTTS, CosyVoice, Fish Speech, Parler-TTS, XTTS, GPT-SoVITS, F5-TTS, Qwen3-TTS, GLM-TTS, Index-TTS, MaskGCT, and more
• 🔌 Plugin Architecture — Add new models with @register_model decorator
• 🚀 Hot-Swap — Switch models at runtime without restarting
• 🌍 Multi-Language — Chinese, English, Japanese, Korean, and more
• 🎯 Multi-Task — Speech synthesis, voice cloning, emotion control, style transfer, streaming
• 💻 Local-First — All inference on-device. No APIs. No data leaves your machine.
• 🐍 Modern Python — uv-native packaging, Pydantic configs, rich CLI
• 📦 Zero-Config Auto-Download — GLM-TTS, Index-TTS, RedFire-TTS, CosyVoice, and Fish Speech automatically download repos, dependencies, and weights on first use

📦 Installation

# Clone the repository
git clone https://github.com/vra/modern-tts.git
cd modern-tts

# Sync all dependencies (recommended)
uv sync --all-extras

# Or install specific extras only
uv sync --extra melotts --extra chattts --extra glm --extra index

# Or just core dependencies
uv sync

Python 3.10+ recommended. Some models (e.g. Index-TTS) require specific PyTorch / transformers versions—see per-model notes below.

🚀 Quick Start

from modern_tts import TTSPipeline

# Synthesize with MeloTTS
pipe = TTSPipeline("melotts-zh")
result = pipe("你好世界，这是语音合成测试。")
result.save("output.wav")

# Switch to ChatTTS for emotional speech
pipe.switch_model("chattts")
result = pipe("这是一个带有情感的语音合成。")
result.save("output_emotion.wav")

# Voice cloning with CosyVoice
pipe.switch_model("cosyvoice-300m")
result = pipe("这是克隆的声音。", task="clone", reference_audio="reference.wav")
result.save("cloned.wav")

# Zero-config voice cloning with GLM-TTS (auto-downloads code)
pipe.switch_model("glm-tts")
result = pipe("你好，这是 GLM-TTS 的语音克隆测试。", task="clone", reference_audio="ref.wav")
result.save("glm_cloned.wav")

# Zero-config voice cloning with Index-TTS (auto-downloads code)
pipe.switch_model("index-tts")
result = pipe("你好，这是 Index-TTS 的语音克隆测试。", task="clone", reference_audio="ref.wav")
result.save("index_cloned.wav")

# Zero-config voice cloning with RedFire-TTS (auto-downloads everything)
pipe.switch_model("redfire-tts")
result = pipe("你好，这是 RedFire-TTS 的语音克隆测试。", task="clone", reference_audio="ref.wav")
result.save("redfire_cloned.wav")

🎙️ Supported Models

✅ Ready to use (loadable out-of-the-box)

ZS-VC = Zero-Shot Voice Cloning (requires a reference_audio sample).

⚠️ Auto-downloads repo, but requires user-trained weights

These models auto-clone their official repos and auto-install dependencies, but require user-trained model weights for specific voices. They are voice cloning training frameworks — you must train a voice model first.

❌ Temporarily unavailable

📋 Changelog & API Changes

Latest

New Models

• glm-tts — LLM + Flow Matching zero-shot TTS (Zhipu AI). Merged previous glm-tts-nano-2512 and glm-tts-2512 into a single glm-tts model ID.
• index-tts — Industrial-level multilingual zero-shot voice cloning (IndexTeam).
• redfire-tts — High-quality Chinese/English zero-shot voice cloning (Xiaohongshu/FireRedTeam). Fully auto-download with Python 3.12 + transformers 5.x compatibility.

Zero-Config Auto-Download

• GLM-TTS, Index-TTS, RedFire-TTS, CosyVoice, and Fish Speech no longer require manual environment variables or PYTHONPATH manipulation.
• On first use, the framework automatically:
  1. Clones the official repository to ~/.cache/modern-tts/repos/
  2. Installs all required pip dependencies (including fairseq with Python 3.12 fixes)
  3. Downloads model weights from HuggingFace Hub
  4. Patches upstream code for compatibility (transformers 5.x, WeTextProcessing)
  5. Proceeds with model loading
• You can still override paths via config.extra or environment variables (GLM_TTS_REPO_PATH, INDEX_TTS_REPO_PATH, REDFIRE_TTS_REPO_PATH).

New Infrastructure Modules

• modern_tts.core.hf_hub — HuggingFace Hub download helpers (download_hf_model, get_hf_model_path) so custom-code adapters don't re-implement caching logic.
• modern_tts.core.repo_manager — Generic git repository auto-downloader (ensure_repo, inject_repo_path) used by adapters that depend on upstream code not on PyPI.
• modern_tts.utils.auto_install — Runtime pip dependency installer (ensure_packages, _pip_install) with uv support and pip-name-to-import-name mapping.

Base Class Improvements

• TextToAudioLLMModel.load() now raises a clear NotImplementedError when a subclass has not set PROCESSOR_CLS / MODEL_CLS, signaling that the subclass must override load() for custom loading logic.

Model ID Changes

🏗️ Architecture

Modern TTS is built on three layers:

1. TTSPipeline — Unified user API. Handles text normalization, task dispatch, model lifecycle.
2. TTSModel / TextToAudioLLMModel — Adapter layer. New models often need only 8 lines of config via TextToAudioLLMModel.
3. Backends — Transformers, vLLM, ONNX Runtime.

Adding a New Model

from modern_tts.core.audio_llm import TextToAudioLLMModel
from modern_tts.core.registry import register_model

@register_model("my-tts-1b")
class MyTTS1B(TextToAudioLLMModel):
    HF_PATH = "org/MyTTS-1B"
    PROCESSOR_CLS = "transformers.AutoTokenizer"
    MODEL_CLS = "transformers.AutoModelForTextToWaveform"
    SUPPORTED_LANGUAGES = {"zh", "en"}
    DEFAULT_SAMPLE_RATE = 24000

    @property
    def model_id(self) -> str:
        return "my-tts-1b"

That's it. The registry auto-discovers it at runtime.

🤝 Contributing

See Contributing Guide for development setup, code style, and PR checklist.

📄 License

Apache-2.0