silicon-asr

Fast local speech-to-text on Apple silicon, from Python. Runs the FluidInference CoreML conversions of NVIDIA's Parakeet models on the Apple Neural Engine — roughly 2x faster than the MLX equivalents at matched accuracy, at a fraction of the power — plus MLX baselines behind the same interface, and a benchmark CLI to measure all of them on your own machine and audio.

Why this exists: the CoreML models are excellent but only had Swift (and Node/Rust) bindings — the maintainer has no plans for a Python SDK. This package is the missing Python path: model download, the TDT/CTC decode loops, long-audio windowing with silence-aware stitching, timestamps, and SRT output.

uv tool install silicon-asr        # or: pip install silicon-asr

silicon-asr transcribe movie.mp4 --srt movie.srt
silicon-asr bench movie.mp4 --runners parakeet-coreml,parakeet-mlx

from silicon_asr.runners import create_runner

runner = create_runner("parakeet-coreml")      # downloads once from the Hub
result = runner.transcribe_file("movie.mp4")   # any format ffmpeg reads
for seg in result.segments:
    print(f"[{seg.start:7.2f} → {seg.end:7.2f}] {seg.text}")

Runners

Models download from Hugging Face on first use (0.6–1.2 GB). The very first CoreML load also compiles for the ANE (~40 s, once per model — the OS caches it; later loads take ~0.2 s).

Benchmarks

Measured on an M3 Max, 12 minutes of speech-dense audio, warm model caches, best-of runs (first-ever load adds one-time ANE compilation, see above):

*The test script's quirks ("VLCaption" spoken as one word) put a ~0.09 WER floor under every runner here — read the deltas between rows, not the absolute values. The 0.6B CoreML rows trade ~1-2 WER points (15-second window stitching vs MLX's 120-second chunks) for ~1.7x speed; CTC-110M trades accuracy for another 7x on top.

Reproduce with silicon-asr bench <audio> --runners ... --reference script.txt (WER is case- and punctuation-insensitive word error rate).

Compute-unit notes, from measurement and the literature:

• The FastConformer encoder runs on the ANE (--compute-units auto/ane) or GPU (gpu); the LSTM prediction network has no ANE kernel and is pinned to CPU on purpose.
• On this M3 Max the ANE slightly beats the GPU; reports on M5-class Max chips show the GPU pulling ahead. Benchmark your own machine — that's what the CLI is for.
• The ANE draws single-digit watts vs tens for the GPU — for battery-bound work (long transcriptions on a laptop), ANE is the right default even at speed parity.

How the long-audio decoding works

The CoreML models take fixed 15-second windows. silicon-asr decodes overlapping windows and cuts each overlap at its largest inter-token gap — i.e. at a silence — so words are never split at window boundaries. TDT decoding threads the LSTM state through the fused JointDecision model (argmax + duration folded into the graph), keeping the Python loop cheap; CTC decoding is a single argmax-collapse pass.

Roadmap

• Nemotron 3.5 streaming 0.6B (FluidInference CoreML): true streaming; currently Discord-gated on the Hub + OpenMDW license — pending access.
• WhisperKit CoreML Whisper (Argmax open models): needs a Python KV-cache decoder loop; would make the table cross-family.
• Energy benchmarks: powermetrics-based joules-per-audio-hour column.
• v2 .mlpackage re-export (the v2 repo only ships compiled .mlmodelc).

Credits

• Model weights: NVIDIA Parakeet (CC-BY-4.0)
• CoreML conversions: FluidInference (CC-BY-4.0) — this package would not exist without their work
• MLX baselines: senstella/parakeet-mlx, mlx-whisper

Apple-silicon Macs only (the ANE/CoreML path requires bare-metal arm64 macOS). Needs ffmpeg on PATH for non-WAV inputs.

Development

uv sync --extra dev --extra mlx
make format && make static-checks && make test

License

MIT (this package). Model weights and conversions carry their own licenses (CC-BY-4.0 for the Parakeet family).