Chatterbox MLX - Apple Silicon Optimized TTS

An MLX-optimized fork of Resemble AI's Chatterbox TTS for Apple Silicon, delivering up to 2.4x faster inference.

Installation

pip install chatterbox-mlx

Requirements

• macOS with Apple Silicon (M1/M2/M3/M4)
• Python 3.11+ (tested primarily with 3.11.12. Also tested with 3.12.12)
• ~4GB disk space for model weights

Important: Python must be compiled with lzma support. If you're using pyenv:

# Install xz library first (provides liblzma)
brew install xz

# Then install Python (or reinstall if already installed)
pyenv install 3.11.12  # or your preferred version

If you see an error about ModuleNotFoundError: No module named '_lzma', you need to install xz and reinstall Python.

CLI Usage

Generate speech directly from the terminal:

    # Generate English speech (auto-generated filename):
    chatterbox "Artificial intelligence has made remarkable strides in recent years, particularly in the field of natural language processing."

    # Generate Spanish speech:
    chatterbox "La inteligencia artificial ha logrado avances notables en los últimos años." --lang es

    # Use the --voice flag to provide a reference audio file for voice cloning:
    chatterbox "Artificial intelligence has made remarkable strides in recent years, particularly in the field of natural language processing." --voice speaker.wav

    # Run multilingual benchmark (saves to benchmark_output/)
    chatterbox --benchmark --languages en es

CLI Options

Quick Start

import torchaudio as ta
from chatterbox.tts_mlx import ChatterboxTTSMLX

# Load model (downloads weights automatically on first run). Default is "cpu", choose "hybrid-mlx" for best performance on an Apple Silicon device.
model = ChatterboxTTSMLX.from_pretrained(device="hybrid-mlx")


# Generate speech
text = "Hello! This is Chatterbox running with MLX optimization on Apple Silicon."
wav = model.generate(text)
ta.save("output.wav", wav, model.sr)

# Voice cloning with reference audio
wav = model.generate(
    text,
    audio_prompt_path="reference_voice.wav",
    exaggeration=0.5,  # Emotion intensity (0.0-1.0)
    cfg_weight=0.5,    # Classifier-free guidance
)

Long-Form Audio Generation

For texts longer than ~50 words, use chunked generation:

long_text = """
Your long text here. It can span multiple paragraphs and sentences.
The generate_long method will automatically split it at sentence boundaries,
generate each chunk separately, and crossfade them together seamlessly.
"""

wav = model.generate_long(
    long_text,
    audio_prompt_path="reference_voice.wav",
    chunk_size_words=50,
    overlap_duration=0.1,
)
ta.save("long_output.wav", wav, model.sr)

🙏 Acknowledgements

This project is built on top of the excellent Chatterbox TTS by Resemble AI. I'm deeply grateful for their work in creating and open-sourcing a production-grade, multilingual text-to-speech system under the MIT license.

This fork focuses specifically on MLX optimizations for Apple Silicon. If you're looking for the original project with CUDA support and the full feature set, please visit the official Resemble AI repository.

What's Different in This Fork?

This package provides native MLX acceleration for Apple Silicon Macs, achieving significant performance improvements:

Key Optimizations

• MLX-Native T3 Model: The 520M parameter Llama 3 backbone runs entirely on MLX
• Float16 KV Cache: Up to 5.8 GB memory savings with 32% faster generation
• Hybrid Architecture: Combines MLX speed with PyTorch quality controls
• Long-Form Generation: Intelligent chunking with crossfade for extended audio

Benchmark Results

All benchmarks run on Apple M4 (32GB RAM), macOS 15.4, Python 3.11, PyTorch 2.8.0.

English TTS Performance

Key findings:

• Hybrid-MLX recommended for production (best quality/speed balance)
• Pure MLX fastest for short texts, but quality degrades on long texts
• 2.2-2.4x speedup vs CPU baseline across all text lengths

Multilingual Performance

Visual Comparison

                    GENERATION TIME COMPARISON

     Short (5 words)
     ├─ CPU        ████████████████████████████████████████ 8.91s
     ├─ Hybrid-MLX ██████████████████ 4.08s (2.2x faster)
     └─ Pure MLX   ████████████████ 3.70s (2.4x faster)

     Medium (31 words)
     ├─ CPU        ████████████████████████████████████████ 57.51s
     ├─ Hybrid-MLX █████████████████ 25.24s (2.3x faster)
     └─ Pure MLX   ████████████████ 24.40s (2.4x faster)

     Long (94 words)
     ├─ CPU        ████████████████████████████████████████ 137.92s
     ├─ Hybrid-MLX █████████████████ 62.66s (2.2x faster)  ✓ Best quality
     └─ Pure MLX   ██████████████████ 68.82s (2.0x faster)

Backend Comparison

RTF = Real-Time Factor (audio_duration / generation_time). Higher is better.

Running Benchmarks

You can reproduce these benchmarks on your own hardware.

English TTS Benchmark

# Full benchmark (all backends)
python benchmark_mps.py --runs 3 --validate

# Quick test with Hybrid-MLX only
python benchmark_mps.py --hybrid-mlx-only --runs 1

# CPU baseline only
python benchmark_mps.py --cpu-only --runs 1

# With voice cloning
python benchmark_mps.py --audio-prompt speaker.wav --runs 3

# Enable memory debugging
DEBUG_MEMORY=1 python benchmark_mps.py --hybrid-mlx-only

Options:

Multilingual Benchmark

# Test specific languages
python benchmark_multilingual.py \
    --audio-prompt speaker.wav \
    --languages en es fr de ja zh \
    --runs 3

# Quick test with Hybrid-MLX
python benchmark_multilingual.py \
    --audio-prompt speaker.wav \
    --languages en es \
    --hybrid-mlx-only

# With validation
python benchmark_multilingual.py \
    --audio-prompt speaker.wav \
    --languages en es fr \
    --validate

Supported Languages: en (English), es (Spanish), fr (French), de (German), it (Italian), pt (Portuguese), ru (Russian), ja (Japanese), zh (Chinese), ko (Korean), ar (Arabic), hi (Hindi), tr (Turkish), pl (Polish), nl (Dutch), sv (Swedish), da (Danish), no (Norwegian), fi (Finnish), el (Greek), he (Hebrew), ms (Malay), sw (Swahili)

Benchmark Output

Results are saved to:

• benchmark_output/benchmark_results.json - English TTS results
• benchmark_multilingual_output/multilingual_results.json - Multilingual results
• Generated audio files: {device}_{category}.wav

Architecture

Chatterbox is a two-stage TTS pipeline. This fork accelerates the most compute-intensive component (T3) with MLX:

┌─────────────────────────────────────────────────────────────────────┐
│                     CHATTERBOX MLX PIPELINE                         │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────────────┐   │
│  │ VoiceEncoder │    │     T3       │    │       S3Gen          │   │
│  │  (PyTorch)   │───▶│    (MLX)     │───▶│   (PyTorch/MPS)      │   │
│  │   ~2M params │    │  520M params │    │     ~80M params      │   │
│  └──────────────┘    └──────────────┘    └──────────────────────┘   │
│                            ▲                                        │
│                            │                                        │
│                    2.4x faster with MLX                             │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Supported Languages

All 23 languages from the original Chatterbox are supported:

Arabic • Danish • German • Greek • English • Spanish • Finnish • French • Hebrew • Hindi • Italian • Japanese • Korean • Malay • Dutch • Norwegian • Polish • Portuguese • Russian • Swedish • Swahili • Turkish • Chinese

from chatterbox.mtl_tts_mlx import ChatterboxMultilingualTTSMLX

model = ChatterboxMultilingualTTSMLX.from_pretrained(device="mps")

# French
wav = model.generate("Bonjour, comment ça va?", language_id="fr")

# Japanese
wav = model.generate("こんにちは、元気ですか？", language_id="ja")

Tips for Best Results

General Use

• Default settings (exaggeration=0.5, cfg_weight=0.5) work well for most cases
• Ensure reference audio matches target language to avoid accent transfer

Expressive Speech

• Lower cfg_weight (~0.3) + higher exaggeration (~0.7) for dramatic delivery
• Higher exaggeration speeds up speech; lower cfg_weight compensates

Memory Usage

Enable debug logging to monitor memory:

DEBUG_MEMORY=1 python your_script.py

Differences from Original Chatterbox

Credits & Links

• Original Project: Resemble AI's Chatterbox
• Resemble AI: resemble.ai - For creating and open-sourcing this incredible TTS system
• Demo: Hugging Face Space
• Evaluation: Outperforms ElevenLabs

Upstream Dependencies

• CosyVoice
• HiFT-GAN
• Llama 3
• MLX

License

MIT License - Same as the original Chatterbox project.

Citation

If you use this project, please cite the original Chatterbox:

@misc{chatterboxtts2025,
  author       = {{Resemble AI}},
  title        = {{Chatterbox-TTS}},
  year         = {2025},
  howpublished = {\url{https://github.com/resemble-ai/chatterbox}},
  note         = {GitHub repository}
}