qwen3-asr-mnn

Qwen3-ASR (0.6B / 1.7B) speech recognition on pure CPU via MNN — no PyTorch, no transformers, no GPU required at runtime.

• Fast: avg RTF ≈ 0.06 (0.6B) / 0.15 (1.7B) on Apple Silicon — 2.3×–2.6× faster than the HF PyTorch fp32 baseline
• Accurate: token-identical to the HF fp32 reference on standard samples (CER 0.0000 for 0.6B); remaining 1.7B differences match the official HF bf16 inference
• Zero-config CLI: models are downloaded automatically from HuggingFace on first use
• Lightweight: runtime deps are only numpy, soundfile, soxr, huggingface_hub

Models: https://huggingface.co/yunfengwang/Qwen3-ASR-MNN (fp16 weights, exported with MNN llmexport)

Install

pip install qwen3-asr-mnn
# or run directly without installing:
uvx qwen3-asr-mnn audio.wav

Prebuilt native extension: macOS arm64, CPython 3.10–3.13. Other platforms can build from source with native/build_native.sh (requires a local MNN tree).

CLI

qwen3-asr-mnn speech.wav                    # 0.6B (default), auto-download model
qwen3-asr-mnn speech.wav --model 1.7b       # higher accuracy
qwen3-asr-mnn speech.wav --json             # full result (ids, rtf, timing)
qwen3-asr-mnn https://example.com/a.wav     # URL input

Any common sample rate / channel layout is accepted (auto-resampled to 16 kHz mono).

Python API

The engine is loaded once and reused — no repeated initialization cost:

from qwen3_asr_mnn import Qwen3ASR

asr = Qwen3ASR(model="0.6b")        # downloads MNN model on first construction

text = asr.transcribe("a.wav")      # fast, engine already warm
text = asr.transcribe("b.wav")

# details: language, token ids, RTF, encoder/prefill/decode timings
r = asr.transcribe_result("a.wav")
print(r["text"], r["language"], f"RTF={r['rtf']:.3f}")

# numpy waveform input (e.g. streaming)
import numpy as np
wav = np.zeros(16000, dtype=np.float32)
print(asr.transcribe(wav, sample_rate=16000))

# precomputed log-mel [128, T] input
from qwen3_asr_mnn import extract_mel_from
mel, valid = extract_mel_from("a.wav")
print(asr.transcribe_mel(mel, valid)["text"])

Qwen3ASR parameters: model ('0.6b'/'1.7b'), model_dir (local dir or HF repo id; defaults to auto-download), threads (default 8), precision ('low' fp16 math — default, token-identical to 'normal' fp32 math but ~2× faster), max_new_tokens.

How it works

audio ──▶ log-mel (numpy STFT, HF-aligned) ──▶ audio_encoder.mnn (windowed, 104-frame static graph)
      ──▶ prompt splice (text embeddings + audio embeddings) ──▶ llm.mnn (KV-cache, fp16, greedy)
      ──▶ text

The audio encoder uses 8×13=104-frame attention windows with frame-level key masking, mathematically equivalent to HF's packed cu_seqlens attention (max error 0 vs HF in fp32). The LLM decoder is exported with MNN llmexport --quant_bit 16 and driven by the MNN Transformer::Llm C++ engine.

License

Apache-2.0