mlx-audio-io

mlx-audio-io is the audio data layer for MLX: fast file decode/encode directly to and from mlx.core.array, with one API across macOS and Linux.

Why This Project Exists

MLX has strong tensor and model primitives, but it does not ship a first-class, cross-platform audio file I/O layer comparable to what torchaudio provides in the PyTorch ecosystem.

In practice, MLX users often end up with one of these compromises:

• bridge through NumPy/SoundFile/librosa with extra copies and inconsistent format behavior
• shell out to ffmpeg/ffprobe for non-WAV workflows
• pull in parts of the PyTorch audio stack just to handle common audio containers/codecs

mlx-audio-io closes that gap with a native backend designed for MLX workloads:

• direct decode/encode into mlx.core.array
• one Python API (load, save, info, stream, batch_load, supports_soxr) on both macOS and Linux
• consistent validation and error messages across platforms
• support for training/inference data access patterns (partial reads, chunked streaming, optional resampling)

Platform Backends

• macOS backend optimized for Apple Silicon via AudioToolbox
• Linux backend with native WAV/MP3 fast paths plus libav-backed codec support (FLAC/M4A/AIFF/CAF)

The public Python API is the same on both platforms: load, save, info, stream, batch_load, supports_soxr.

Backend Feature Matrix

Unsupported format/encoding combinations fail with explicit ValueError messages.

Installation

End users (PyPI)

For normal use:

pip install mlx-audio-io

MLX compatibility

mlx-audio-io is published as an sdist, so pip install compiles the extension locally. Since 1.3.23 the extension does not build against MLX's C++ headers or link libmlx: decoded audio reaches MLX as a DLPack capsule (mx.from_dlpack), and save()/resample() read MLX arrays through the buffer protocol. The build therefore needs no MLX at all, and one build works with any installed MLX from 0.32.0 on (the first release with mx.from_dlpack):

• macOS: mlx>=0.32.0
• Linux: mlx[cpu]>=0.32.0

Upgrading MLX does not require rebuilding mlx-audio-io. Releases up to 1.3.22 linked libmlx and had to be rebuilt for each MLX version; consumers no longer need the [tool.uv.extra-build-dependencies] match-runtime entry those releases asked for.

Contributors (source checkout)

For local development and tests:

git clone https://github.com/ssmall256/mlx-audio-io.git
cd mlx-audio-io
uv sync --extra dev

To create a standalone sdist or wheel, use uv build.

Hard Rule: Do Not Copy .venv Between Machines

Do not copy project virtual environments across machines. Native extensions can fail integrity/code-sign checks or crash when moved between hosts.

If you already copied one, recreate it:

rm -rf .venv && uv venv --python 3.11 && uv sync

Linux source build behavior

Linux source builds require libav and use direct libav-backed paths:

• Linux info() for non-WAV formats uses direct libav metadata.
• Linux load() for non-WAV formats uses direct libav decode for all offset/duration combinations.
• Linux stream() for non-WAV formats uses direct libav packet/frame decode.
• Linux save() for encoded formats (.mp3, .flac, .m4a, .aiff/.aif, .caf) uses direct libav encode/mux.

Wheel portability and libsoxr

• soxr_hq / soxr_vhq are optional and enabled only when libsoxr is available at build time.
• macOS wheel builds are automatically repaired with delocate in this project build backend, so external libsoxr dylibs are bundled into the wheel instead of relying on Homebrew paths.
• Linux wheel repair via auditwheel is supported and can be enabled with MLX_AUDIO_IO_REPAIR_LINUX=1 in release CI.
• To disable wheel repair explicitly (not recommended for release builds), set MLX_AUDIO_IO_REPAIR_WHEEL=0.
• Release CI verifies wheel linkage with tools/check_wheel_linkage.py and fails if absolute host library paths are detected.
• Third-party notices for bundled libsoxr are shipped in-package at:
  • mlx_audio_io/THIRD_PARTY_NOTICES.md
  • mlx_audio_io/licenses/libsoxr/
• Current release publish workflows build and publish sdist artifacts; a separate macOS wheel job verifies wheel linkage and notice packaging as a release gate.

Requirements

• Python 3.10+
• Runtime:
  • macOS: Apple Silicon + mlx
  • Linux: mlx[cpu] (current default)
• Source builds:
  • CMake 3.24+, C++20 toolchain, pkg-config
  • Linux default build: libavformat-dev, libavcodec-dev, libavutil-dev, libswresample-dev

Troubleshooting

• ModuleNotFoundError: mlx_audio_io
  • Install in the project environment (uv sync) and run via uv run ....
• ImportError for mlx on Linux
  • Ensure Linux dependency is installed as mlx[cpu].
• Build failures on source installs
  • Verify build-essential, cmake, ninja-build, and pkg-config are installed.
• Extended Linux format support errors (.mp3, .m4a, .flac, .aiff, .caf)
  • For default Linux builds, ensure runtime libav libraries are present (libavformat, libavcodec, libavutil, libswresample).
• MP3 test fixture generation failures
  • Tests that generate MP3 fixtures require ffmpeg or lame available on PATH.
• Native import failures or unexpected crashes
  • Run diagnostics: python -m mlx_audio_io.doctor
  • Check MLX runtime compatibility: python -c "import mlx_audio_io as aio; print(aio.show_build_info())"
  • If build_mlx_version and runtime_mlx_version differ, rebuild against the MLX you run — see "Installing: the extension is compiled on your machine". The error the loader prints contains the exact two commands.
  • Moving your runtime instead also works, if nothing else pins it: pip install -U "mlx==<build_mlx_version>"
  • Avoid pip install --no-deps for mlx-audio-io unless you manually pin a matching mlx version.
  • Recreate env (do not copy .venv between machines): rm -rf .venv && uv venv --python 3.11 && uv sync

Quickstart

from mlx_audio_io import load, save, info, stream, batch_load, supports_soxr

# Load
x, sr = load("speech.wav")

# Resample + mono (auto-selects soxr_vhq when available, falls back to "best")
x16, sr16 = load("speech.wav", sr=16000, mono=True)

# Metadata without decoding
meta = info("speech.wav")

# Stream in chunks
for chunk, chunk_sr in stream("long.wav", chunk_duration=2.0):
    pass

# Save WAV
save("out.wav", x, sr)
save("out_pcm16.wav", x, sr, encoding="pcm16")

# Batch load
items = batch_load(["a.wav", "b.wav"], sr=16000, mono=True)

Additional save examples:

save("out.flac", x, sr)
save("out.mp3", x, sr, bitrate="192k")
save("out.m4a", x, sr, bitrate="256k")
save("out.m4a", x, sr, encoding="alac")

API Reference

load

load(path, sr=None, offset=0.0, duration=None, mono=False, mono_mode="mean",
     layout="channels_last", dtype="float32", resample_quality="default",
     low_memory=False, squeeze_mono=False)

Decode audio into an mlx.core.array. Returns (audio, sample_rate).

On macOS, default resampling when sr is specified automatically routes through Apple AudioToolbox (best), delivering ~4.5x faster throughput than soxr_vhq with >85 dB SNR. Set MLX_AUDIO_IO_SOXR_DEFAULT=1 to use libsoxr instead. On Linux WAV/MP3 fast paths, resample quality levels currently map to the same linear behavior. soxr_hq/soxr_vhq use true libsoxr resampling (when built with libsoxr). If soxr_hq/soxr_vhq is requested without libsoxr support, load()/resample() raise RuntimeError. torchaudio_compat requires torch + torchaudio and uses torchaudio.functional.resample.

When sr is specified and resample_quality is left at "default", load() automatically selects best on macOS and soxr_vhq on Linux when libsoxr is available. You can still override explicitly:

audio, sr = load("speech.wav", sr=16000, resample_quality="soxr_hq")

batch_load

batch_load(paths, sr=None, mono=False, mono_mode="mean", dtype="float32", num_workers=4, squeeze_mono=False)

Threaded multi-file load(). Returns list[(audio, sample_rate)].

save

save(path, audio, sr, layout="auto", encoding="float32",
     bitrate="auto", clip=True)

Write audio from mx.array (or numpy.ndarray) to disk.

stream

stream(path, chunk_frames=None, chunk_duration=None, sr=None,
       mono=False, mono_mode="mean", dtype="float32", offset=0.0, duration=None)

Return an iterator yielding (audio_chunk, sample_rate). Exactly one of chunk_frames or chunk_duration is required.

info

info(path)

Return AudioInfo metadata without decoding sample buffers.

supports_soxr

supports_soxr()

Return True when the installed native extension was built with libsoxr support. Use this to select resample_quality at runtime.

Testing

Run all tests:

uv sync --extra dev
uv run python -m pytest -q

Run Linux supported subset:

uv run python -m pytest -q -m "not apple_only"

Run Apple-only subset:

uv run python -m pytest -q -m "apple_only"

Linux Docker run from a macOS host:

docker run --rm -it --platform linux/arm64 \
  -v "$PWD":/work -w /work \
  python:3.14-bookworm bash -lc '
    apt-get update && apt-get install -y --no-install-recommends \
      build-essential cmake ninja-build pkg-config ffmpeg \
      libavformat-dev libavcodec-dev libavutil-dev libswresample-dev &&
    python -m pip install -U pip uv &&
    uv sync --extra dev &&
    uv run python -m pytest -q -m "not apple_only"
  '

Performance

Benchmark methodology, commands, and full result tables live in docs/benchmarking.md.

Headline numbers (194.8s stereo PCM16 WAV @ 44.1 kHz, p50 median latency):

Full tables with torchaudio comparisons, M1 Max, and Linux x86_64 results are in the benchmarking doc.

License

MIT