Skip to main content

mlx-speech

Text-to-speech and speech recognition on Apple Silicon

Voice cloning, audio editing, sound effects, and transcription. All running locally in MLX.

PyPI Python 3.13+ License: MIT CI

Quick start · Models · Model guides · Hugging Face weights · Project website

mlx-speech is an open-source Python library for text-to-speech (TTS) and automatic speech recognition (ASR) on Apple Silicon Macs. Models share a Python API and command-line interface, with inference implemented in Apple's MLX framework.

Audio processing stays on your Mac. Inference needs neither PyTorch nor a cloud service. Published weights download on first use, and local checkpoint paths support offline loading.

Installation

Requires an Apple Silicon Mac (M1 or later) and Python 3.13+.

pip install mlx-speech

Quick Start

Python:

import mlx_speech
from mlx_speech.audio import write_wav

# Text-to-speech
model = mlx_speech.tts.load("fish-s2-pro")
result = model.generate("Hello from mlx-speech!")
write_wav("output.wav", result.waveform, sample_rate=result.sample_rate)

# Speech-to-text
asr = mlx_speech.asr.load("qwen3-asr-1.7b")
print(asr.generate("audio.wav").text)

CLI:

mlx-speech tts --model fish-s2-pro --text "Hello!" -o output.wav
mlx-speech asr --model qwen3-asr-1.7b --audio speech.wav

Models

Pass the selector to tts.load(), asr.load(), or --model. Names link to guides; weight links open the Hugging Face repositories.

Text-to-speech, voice cloning, and sound effects

Model Use it for Selector Weights
Fish S2 Pro Voice cloning and emotion tags fish-s2-pro int8
VibeVoice Large Speech synthesis and voice cloning vibevoice int8
LongCat AudioDiT Diffusion speech synthesis longcat int8
OpenMOSS TTS Local Speech synthesis and voice cloning moss-local int8
MOSS-TTSD Multi-speaker dialogue moss-ttsd int8
OpenMOSS Sound Effect Sound effects from text moss-sound-effect 4-bit
Step-Audio-EditX Voice cloning and audio editing step-audio int8
DramaBox Speech synthesis in 48 kHz stereo dramabox BF16¹
dots.tts SOAR Voice cloning and waveform streaming dots-tts-soar int8 + base
dots.tts MeanFlow Distilled TTS and waveform streaming dots-tts-mf int8 + base
FireRedTTS3 Base Multilingual voice cloning at 24 kHz fireredtts3-base BF16
Breeze TTS 2 Voice design, instruction direction, and cloning with waveform streaming local path BF16, converted locally²

Speech-to-text

Model Use it for Selector Weights
Cohere Transcribe Multilingual transcription cohere-asr int8
Qwen3-ASR-1.7B English, Chinese, and mixed speech qwen3-asr-1.7b int8 · BF16
Confucius4-R2T2 Real-time streaming transcription, 30 languages confucius4-r2t2 BF16
NVIDIA Nemotron 3.5 ASR Streaming Multilingual streaming transcription nemotron-asr-streaming int8
IBM Granite Speech 4.0 1B Speech recognition with a selective-int8 language model granite-speech-4.0-1b int8
Loading local weights, shared repositories, DramaBox components, and Breeze TTS 2

Flat model repositories accept an alias or a full repository ID. tts.load("fish-s2-pro") and tts.load("appautomaton/fishaudio-s2-pro-8bit-mlx") are equivalent. For a repository containing multiple artifacts, use an alias or specify artifact_subdir. Original checkpoint paths work where the model guide documents their layout.

¹ DramaBox also downloads the Gemma 3 12B text encoder automatically. Its optional denoise_ref=True setting uses the MLX RE-USE / SEMamba enhancer to clean noisy voice references. Denoising is off by default, and the enhancer weights carry the NSCLv1 non-commercial license. The DramaBox guide covers these components and advanced controls.

² Breeze TTS 2 has no published MLX weights or alias yet. Download the official checkpoint, convert it with scripts/convert/breeze_tts.py, and pass the converted directory to tts.load(). The weights and outputs made from them carry the BreezeBlue non-commercial license. The Breeze TTS 2 guide covers conversion, controls, and known limits.

More examples

Python: voice cloning, streaming transcription, and model discovery

Voice cloning with emotion tags

import mlx_speech
from mlx_speech.audio import write_wav

model = mlx_speech.tts.load("fish-s2-pro")
result = model.generate(
    "[excited] This is amazing!",
    reference_audio="reference.wav",
    reference_text="Transcript of the reference audio.",
)
write_wav("cloned.wav", result.waveform, sample_rate=result.sample_rate)

Live transcription with Confucius4-R2T2

import mlx_speech

asr = mlx_speech.asr.load("confucius4-r2t2")
session = asr.stream_session(language="English")
for pcm in microphone:                 # float32, 16 kHz mono
    print(session.feed(pcm).committed)
print(session.finalize().text)

Streaming transcription with Nemotron

import mlx_speech
from mlx_speech.audio import load_audio

nemotron = mlx_speech.asr.load("nemotron-asr-streaming")
session = nemotron.stream_session(language="en-US", att_context_size=(56, 3))
waveform, _ = load_audio("audio.wav", sample_rate=16_000, mono=True)
for start in range(0, int(waveform.size), 1_600):
    session.feed(waveform[start : start + 1_600])
session.finalize()
print(session.result().text)

Granite transcription and model discovery

import mlx_speech

# Granite defaults to the published selective-int8 artifact
granite = mlx_speech.asr.load("granite-speech-4.0-1b")
print(granite.generate("audio.wav").text)

# Discover models
mlx_speech.tts.list_models()
mlx_speech.tts.list_models(detailed=True)  # includes shared-repo artifact paths
mlx_speech.asr.list_models()
CLI: waveform streaming, voice cloning, editing, and sound effects
# Bounded waveform streaming with dots.tts
mlx-speech tts --model dots-tts-soar --text "Hello!" --stream -o streamed.wav

# Voice cloning with emotion tags
mlx-speech tts --model fish-s2-pro \
  --text "[whisper] Just between us..." \
  --reference-audio ref.wav \
  --reference-text "Transcript of reference." \
  -o cloned.wav

# Step Audio emotion editing
mlx-speech tts --model step-audio \
  --reference-audio input.wav \
  --reference-text "Transcript." \
  --edit-type emotion --edit-info happy \
  -o happy.wav

# Sound effect generation
mlx-speech tts --model moss-sound-effect \
  --text "rolling thunder with rainfall" \
  --duration-seconds 8 \
  -o thunder.wav

# Transcribe audio with any ASR selector
mlx-speech asr --model cohere-asr --audio speech.wav
mlx-speech asr --model qwen3-asr-1.7b --audio speech.wav --language Chinese

# Local checkpoint paths work anywhere an alias does
mlx-speech tts --model models/fish_s2_pro/mlx-int8 --text "Hello!" -o output.wav

# Discover models
mlx-speech tts --list-models
mlx-speech asr --list-models

For sampling controls, diffusion steps, batch generation, and other advanced options, follow the model's guide. Each guide names the supported controls and the script that exposes them.

Conversion

Use the published weights to get started. To convert an original checkpoint, follow its model guide for source files, precision options, and the matching conversion script.

Conversion runs separately from inference. Tools needed to read source checkpoints are not runtime requirements.

Development

git clone https://github.com/appautomaton/mlx-speech.git
cd mlx-speech
uv sync
uv run pytest
uv run ruff check .

The default suite needs no checkpoints. Real-weight tests are separate; see the testing guide. Website changes belong in the frontend repository; see website ownership.

mlx-speech/
  src/mlx_speech/     library code
  scripts/           conversion, generation, eval, and audit entry points
  models/            local checkpoints (not in git)
  tests/             unit, checkpoint, runtime, integration tests
  docs/              model-family behavior guides

License

Library code is released under the MIT license. Model weights retain their respective licenses, listed in their model cards.

Built and maintained by App Automaton.

Acknowledgements

Thanks to the linux.do community for its inspiration and support.

Metadata

Release files for mlx-speech 0.5.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mlx-speech 0.5.5
File Size Uploaded Provenance Actions
mlx_speech-0.5.5.tar.gz 500.8 kB With provenance Details for mlx_speech-0.5.5.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for mlx-speech 0.5.5
File Interpreter ABI Platform Actions
mlx_speech-0.5.5-py3-none-any.whl Python 3 none any Details for mlx_speech-0.5.5-py3-none-any.whl

Total release size: 1.1 MB

Release files / mlx_speech-0.5.5.tar.gz

Download URL mlx_speech-0.5.5.tar.gz
Size 500.8 kB
Tags Source
SHA-256 checksum
How to use checksums
b4dc32d24371f496baddbb517eeb31957180b52aa1bed58bba76d754ad719134
BLAKE2b-256 checksum
How to use checksums
e47f3600962d1e1a07faed82074cc603ff0094b47d17b2b3e86b13acac5a15c7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Release files / mlx_speech-0.5.5-py3-none-any.whl

Download URL mlx_speech-0.5.5-py3-none-any.whl
Size 630.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a6faa9bace9b0992840a8d18f6ddab7e256df41ce21607792984590b9a009136
BLAKE2b-256 checksum
How to use checksums
3dfe155c506cea97b5c778ff77a3a63f8f242700f467ad0b0e70598d760b9505
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Security for mlx-speech 0.5.5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for mlx_speech-0.5.5.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page