mlx-audio-separator

MLX-native stem separation for Apple Silicon Macs.

This project ports the inference paths from audio-separator (upstream repo: nomadkaraoke/python-audio-separator) to MLX so separation runs on Apple Silicon without requiring PyTorch or ONNX Runtime at inference time. Core runtime components are powered by mlx-audio-io (audio I/O) and mlx-spectro (spectral transforms).

Requirements

• macOS 13+ (Ventura or later)
• Apple Silicon (M1/M2/M3/M4)
• Python 3.10+

Installation

pip install mlx-audio-separator

If you need first-run conversion from upstream checkpoints (.ckpt/.onnx/Demucs weights), install conversion extras:

pip install "mlx-audio-separator[convert]"

Quick Start

CLI

# Separate with default model
mlx-audio-separator song.mp3

# Use a specific model
mlx-audio-separator song.mp3 -m htdemucs_ft.yaml

# List supported models
mlx-audio-separator --list_models

Python

from mlx_audio_separator import Separator

sep = Separator()
sep.load_model()
outputs = sep.separate("song.mp3")
print(outputs)

Supported Architectures

• Roformer (BS-Roformer and MelBand-Roformer families)
• MDXC (including MDX23C-style checkpoints)
• MDX
• VR
• Demucs

Demucs cache security and migration

Demucs conversion stores MLX weights as <model>.safetensors with a validated <model>_config.json sidecar. The loader verifies that the two files match before constructing a model. Legacy <model>_mlx.pkl caches are never deserialized. When automatic conversion is enabled they are left untouched and replaced by newly generated safe files from the official Demucs source.

To regenerate a cache explicitly:

pip install "mlx-audio-separator[convert]"
python -m mlx_audio_separator.demucs_mlx.mlx_convert htdemucs \
  --output-dir ~/.cache/demucs-mlx

Official Demucs downloads retain their filename hash checks and are loaded with PyTorch's restricted weight-only deserializer. This trusts the installed PyTorch, Demucs, NumPy, and optional DiffQ implementations plus the official model registry; arbitrary checkpoint globals and local pickle caches are not trusted.

Validation Snapshot

Release validation snapshot (2026-02-24 to 2026-02-26):

Scope: Apple Silicon (M4 mini), MUSDB18-HQ test subset, release gate + parity smoke model set.

Detailed evidence and provenance: docs/release-validation.md.

Performance Snapshot

MLX vs audio-separator (ABBA, 12-song MUSDB18-HQ test subset, M4 mini):

Median speedup across the 4-model overlap set: 1.847x.

These numbers are scoped to the benchmark settings above and are not universal guarantees for all machines, models, or audio inputs.

Stable Runtime Tuning

Release-facing stable controls:

• --speed_mode {default,latency_safe,latency_safe_v2,latency_safe_v3}
• --cache_clear_policy {aggressive,deferred}
• --write_workers <int>

Example:

mlx-audio-separator song.mp3 \
  --speed_mode latency_safe \
  --cache_clear_policy deferred \
  --write_workers 2

Basic benchmark command:

mlx-audio-separator \
  --benchmark song.mp3 \
  --benchmark_warmup 1 \
  --benchmark_repeats 3 \
  --benchmark_profile

BS-Roformer-SW Performance (Opt-In)

For BS-Roformer-SW.ckpt, use the opt-in no-drift FLAC profile:

mlx-audio-separator song.mp3 \
  -m BS-Roformer-SW.ckpt \
  --output_format FLAC \
  --speed_mode latency_safe_v3

latency_safe_v3 keeps model inference behavior conservative and focuses on safe end-to-end latency wins (deferred cache clearing + async stem writes).

To avoid repeated checkpoint conversion overhead, pre-convert once to *.safetensors and exit:

mlx-audio-separator \
  -m BS-Roformer-SW.ckpt \
  --save_converted_safetensors \
  --preconvert_only

safetensors primarily improves model load/startup time. It is not expected to materially change per-file inference latency.

Validation command (latency + deterministic equivalence):

uv run --with torch python scripts/perf/compare_latency.py \
  --corpus-file /tmp/corpus_one.txt \
  --baseline-config scripts/perf/configs/bs_roformer_sw_default_baseline.json \
  --candidate-config scripts/perf/configs/bs_roformer_sw_latency_safe_v3_candidate.json \
  --model-file-dir /tmp/audio-separator-models \
  --allow-speed-mode-mismatch \
  --target-improvement-demucs-mdxc 10.0 \
  --equivalence-check \
  --equivalence-threshold-rel-l2 1e-6 \
  --output-json /tmp/bs_roformer_sw_latency_safe_v3_compare.json \
  --output-markdown /tmp/bs_roformer_sw_latency_safe_v3_compare.md

BS-Roformer-SW Optimization Program (Opt-In Tracks)

As of March 4, 2026, latency_safe_v3 remains the only promoted runtime win for BS-Roformer-SW.ckpt; all experimental tracks below are parked pending new evidence.

Candidate configs for staged exploration live under scripts/perf/configs/:

• bs_roformer_sw_cand_grouped_bandmask.json
• bs_roformer_sw_cand_fused_ola.json
• bs_roformer_sw_cand_stream_pipeline.json
• bs_roformer_sw_cand_compile_fullgraph.json
• bs_roformer_sw_cand_flac_fastwrite.json

Corpus manifest templates:

• Quick gate (3 files): scripts/perf/corpora/bs_roformer_sw_quick.txt
• Full gate (12 files): scripts/perf/corpora/bs_roformer_sw_full.txt

Quick-gate example:

uv run --with torch python scripts/perf/compare_latency.py \
  --corpus-file scripts/perf/corpora/bs_roformer_sw_quick.txt \
  --baseline-config scripts/perf/configs/bs_roformer_sw_latency_safe_v3_baseline.json \
  --candidate-config scripts/perf/configs/bs_roformer_sw_cand_grouped_bandmask.json \
  --model-file-dir /tmp/audio-separator-models \
  --target-improvement-demucs-mdxc 3.0 \
  --equivalence-check \
  --equivalence-threshold-rel-l2 1e-6 \
  --equivalence-max-files 1 \
  --output-json /tmp/bs_roformer_sw_quick_gate.json \
  --output-markdown /tmp/bs_roformer_sw_quick_gate.md

Full-gate example:

uv run --with torch python scripts/perf/compare_latency.py \
  --corpus-file scripts/perf/corpora/bs_roformer_sw_full.txt \
  --baseline-config scripts/perf/configs/bs_roformer_sw_latency_safe_v3_baseline.json \
  --candidate-config scripts/perf/configs/bs_roformer_sw_cand_grouped_bandmask.json \
  --model-file-dir /tmp/audio-separator-models \
  --target-improvement-demucs-mdxc 5.0 \
  --equivalence-check \
  --equivalence-threshold-rel-l2 1e-6 \
  --equivalence-max-files 0 \
  --output-json /tmp/bs_roformer_sw_full_gate.json \
  --output-markdown /tmp/bs_roformer_sw_full_gate.md

Documentation

License

This project is MIT licensed.

Acknowledgments

mlx-audio-separator is derived from audio-separator (upstream repo: nomadkaraoke/python-audio-separator, MIT) by beveradb and the nomadkaraoke community. Substantial portions of the architecture, model loading, and separation logic are adapted from that project. If you find this package useful, please also star and support the upstream project.

The models used by this project were trained by the Ultimate Vocal Remover community, primarily @Anjok07 and @aufr33. See THIRD_PARTY_NOTICES.md for full attribution and license details.

Additional references:

• mlx-audio-io
• mlx-spectro
• BS-Roformer
• Demucs (Meta Research)
• Ultimate Vocal Remover
• audio-separator / nomadkaraoke/python-audio-separator