demucs-mlx

Split any song into its individual stems — vocals, drums, bass, and other instruments — directly on your Mac.

demucs-mlx is a fast, native Apple Silicon port of Meta's Demucs music source separation model, built on MLX. No PyTorch required.

Features

• Up to 96x realtime on Apple Silicon (40-core M4 Max, default settings) — >3.3x faster than Demucs with PyTorch MPS
• Auto-tuned hardware topology — automatically configures batch sizes to match Apple Silicon memory bandwidth, GPU cores, and SLC cache
• Matches upstream Demucs for every registry model: 72–83 dB SNR per stem against PyTorch on the same input, checked in CI
• Custom fused Metal kernels (GroupNorm+GELU, GroupNorm+GLU, zero-transpose GLU, OLA)
• Metal-free fallbacks for non-Apple platforms (Linux)
• No PyTorch required at inference time
• Automatic resampling — input files at any sample rate are resampled to the model rate
• Audio I/O via mlx-audio-io
• STFT/iSTFT via mlx-spectro

Requirements

• Python >= 3.10
• macOS with Apple Silicon (recommended) or Linux with MLX
• MLX 0.31.2 to 0.32.x, with mlx-audio-io 1.3.x and mlx-spectro 0.9.3 or newer

Install

pip install demucs-mlx

On first run, demucs-mlx loads cached MLX weights if available. If the optional mlx-weights package is installed locally, demucs-mlx uses its shared cache. Otherwise it uses its built-in cache. A cache miss is converted internally from the official Demucs registry using the restricted loader described below.

To bootstrap a missing model with the public package, install the conversion extra:

pip install 'demucs-mlx[convert]'

You can explicitly generate a safe cache in any directory with:

python -m demucs_mlx.mlx_convert htdemucs --output-dir ~/.cache/demucs-mlx

Once weights are cached, the convert extra is no longer needed for inference.

CLI usage

demucs-mlx /path/to/audio.wav

Options:

-n, --name          Model name (default: htdemucs)
-o, --out           Output directory (default: separated)
--shifts            Number of random shifts (default: 1)
--seed              Optional RNG seed for reproducible shifts (default: none)
--overlap           Overlap ratio (default: 0.25)
-b, --batch-size    Batch size (default: auto, matched to hardware topology)
--compile           Opt in to whole-forward compilation for fixed shapes
--attention         Attention kernel precision: fp16 (default) or fp32 (~3% slower, same accuracy)
--write-workers     Concurrent writer threads (default: 2)
--ane-time-encoder  Offload the first HTDemucs waveform convolution to the Neural Engine
--stem              For htdemucs_ft, compute only drums, bass, other, or vocals
--list-models       List available models
-v, --verbose       Verbose logging

Python usage

from demucs_mlx import Separator

separator = Separator()
origin, stems = separator.separate_audio_file("song.wav")

# stems is a dict: {"drums": array, "bass": array, "other": array, "vocals": array}
for name, audio in stems.items():
    print(f"{name}: {audio.shape}")

To keep outputs as MLX arrays (avoids GPU-to-CPU copy):

origin, stems = separator.separate_audio_file("song.wav", return_mx=True)

For reproducible shift sampling (while keeping shifts=1 behavior), pass a seed:

separator = Separator(model="htdemucs", shifts=1, seed=0)
origin, stems = separator.separate_audio_file("song.wav")

Tuning

The defaults are the recommended configuration. You should not need to set anything to get the best results; this section exists so the levers are discoverable rather than buried in source.

Environment variables

Version history

See CHANGELOG.md, which the release workflow reads directly.

Performance

Benchmarked on a 3:15 stereo track (44.1 kHz, 16-bit) using htdemucs with default settings:

Apple M4 Max, 128 GB. All runs use htdemucs with default settings and a single warm-up pass before timing.

In a direct alternating comparison, the current development branch's three GPU optimizations together reduced default htdemucs separation time by 24–27%, increasing audio throughput by 31–38%. These are synthetic-input, loaded-model measurements that exclude file I/O; the comparison job had substantial background GPU use, so its absolute times should not be compared with the track benchmark above. See the reproducible throughput measurements for the paired results and fidelity.

Models

For a single fine-tuned stem, --stem runs only its specialized model:

demucs-mlx -n htdemucs_ft --stem vocals song.wav
demucs-mlx -n htdemucs_ft --stem drums song.wav
demucs-mlx -n htdemucs_ft --stem bass song.wav

The full four-stem htdemucs_ft run remains available by omitting --stem.

MLX model cache

Pre-converted MLX weights are cached under ~/.cache/demucs-mlx by default. When the optional mlx-weights package is installed, demucs-mlx uses its shared ~/.cache/mlx-weights/demucs-mlx directory instead.

Cache format v1 consists of <model>.safetensors and a versioned <model>_config.json sidecar. Arrays are saved and loaded with MLX's native safetensors support. The bounded JSON metadata records the exact MLX model classes and constructor data, ensemble shape and weights, ordered official Demucs source signatures/checksums, conversion time, actual MLX version, verification result, and the SHA-256 of the safetensors file. Exceptional constructor values such as Fraction use a narrowly validated tagged JSON representation. The digest and complete metadata are validated before arrays are loaded or a model is constructed.

Older <model>_mlx.pkl files are unsafe legacy caches. demucs-mlx never opens, rewrites, or deletes them. If conversion dependencies are installed, a legacy-only cache is ignored and safe v1 artifacts are regenerated from the verified official source. Without automatic conversion, the error includes the ignored pickle path and the exact regeneration command. Partial, corrupt, unversioned, or otherwise invalid safetensors/config pairs fail closed and never fall back to a pickle; move those safe artifacts aside and run, for example:

python -m demucs_mlx.mlx_convert htdemucs --output-dir ~/.cache/demucs-mlx

Model trust boundary

Conversion requires PyTorch 2.6 or newer before any checkpoint is downloaded or deserialized. Official packages retain filename-hash verification and are loaded with weights_only=True plus a scoped allowlist of exact Demucs classes and narrowly needed compatibility types. The package shape, exact model class, constructors, and ordinary or quantized state are validated before trusted Demucs code constructs a model. There is no unrestricted fallback.

Installed PyTorch, Demucs, NumPy, optional DiffQ quantization code, MLX, and the packaged official model registry are inside the trust boundary. Arbitrary checkpoint globals and local pickle caches are not trusted. Restricted loading prevents executable pickle globals; it is not a resource-exhaustion sandbox for otherwise valid tensor files.

Documentation

• API reference: docs/api.md
• Development workflow: docs/development.md
• Platform notes: docs/platform.md
• Neural Engine waveform prototype and measured results: docs/ane-prototype.md
• Throughput experiments and GroupNorm speedup: docs/throughput.md

License

MIT. Based on Demucs by Meta Research. See LICENSE for details.