qwen3-asr-mlx

Qwen3-ASR speech-to-text inference on Apple Silicon via MLX.

An MLX implementation of the Qwen3-ASR 0.6B and 1.7B pipelines, with no PyTorch or transformers dependency.

This package provides inference code only. Model weights are developed by Qwen Team, Alibaba Cloud under the Apache 2.0 license and downloaded separately from HuggingFace Hub on first use.

Quick Start

Apple Silicon required. macOS 14+, Python 3.10+, MLX 0.31+.

pip install qwen3-asr-mlx

from qwen3_asr_mlx import Qwen3ASR

model = Qwen3ASR.from_pretrained("mlx-community/Qwen3-ASR-1.7B-bf16")
result = model.transcribe(
    "audio.wav",
    context="Vocabulary: Quilter, apostle, gospel.",
)
print(result.text)      # "Hello, world."
print(result.language)  # "English"
print(result.duration)  # 3.2

Model weights download automatically from HuggingFace Hub on first use.

The 1.7B variant gives the best transcription quality and is the recommended default. For lower memory use and latency, load mlx-community/Qwen3-ASR-0.6B-bf16 instead.

Features

• On-device via MLX. No server, no network during inference.
• No PyTorch or transformers dependency.
• Audio formats via soundfile (WAV, FLAC, MP3, etc.)
• Reference-exact features: the log-mel frontend matches the Whisper feature extractor Qwen3-ASR was trained with.
• Automatic language detection from model output.
• Context and hotwords through Qwen3-ASR's official system-prompt format.
• Greedy and sampling-based decoding with repetition penalty, top-k, and nucleus sampling.
• Thread-safe with internal lock for concurrent callers.
• Long audio support with automatic chunking at low-energy boundaries (up to 20 minutes).
• Context manager for resource cleanup.

API

Qwen3ASR.from_pretrained(model_id_or_path, **kwargs)

Load a model from a local directory or the HuggingFace Hub.

model = Qwen3ASR.from_pretrained("mlx-community/Qwen3-ASR-1.7B-bf16")
# or from a local path
model = Qwen3ASR.from_pretrained("/path/to/model")

model.transcribe(audio, **kwargs) -> TranscriptionResult

Transcribe audio to text.

model.warm_up()

Run a short dummy inference to pre-compile the MLX compute graph. Eliminates the latency spike on the first real transcription.

model.close()

Release model weights and free memory. Called automatically when using the context manager.

with Qwen3ASR.from_pretrained("mlx-community/Qwen3-ASR-1.7B-bf16") as model:
    result = model.transcribe("audio.wav")

TranscriptionResult

@dataclass
class TranscriptionResult:
    text: str        # clean transcription
    language: str    # detected language (e.g. "English")
    duration: float  # audio duration in seconds

Architecture

Audio (WAV/FLAC/MP3/ndarray)
  │
  ▼
Mel Spectrogram (128-bin, numpy STFT, Slaney filterbank)
  │
  ▼
Audio Encoder (Conv2D stem → 24-layer transformer → projection)
  │
  ▼
Text Decoder (28-layer Qwen3, GQA 16Q/8KV, QK-norm, SwiGLU, RoPE)
  │
  ▼
TranscriptionResult { text, language, duration }

Requirements

• Apple Silicon Mac (M1 or later)
• macOS 14+ (the oldest macOS MLX ships wheels for)
• Python 3.10+
• MLX 0.31+

The MLX conversion and this runtime are community maintained; they are not Qwen's official PyTorch/Transformers stack. The 1.7B model generally produces higher-quality transcripts, but needs more unified memory and has higher latency than the 0.6B model.

Development

git clone https://github.com/gabrimatic/qwen3-asr-mlx.git
cd qwen3-asr-mlx
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
python -m pytest -v
ruff check src tests && ruff format --check src tests
mypy

Skip model-loading tests with -m "not slow".

Credits

Qwen3-ASR by Qwen Team, Alibaba Cloud · MLX by Apple · Model weights from mlx-community

License

This inference code is released under the MIT License. See LICENSE for details.

The model weights have their own license (Apache 2.0). See Model License above.

Created by Soroush Yousefpour