mlx-meralion

MLX-native inference for MERaLiON AudioLLM on Apple Silicon.

MERaLiON is A*STAR's multimodal audio-language model for speech transcription, translation, spoken question answering, and more.

Installation

pip install mlx-meralion

Requires macOS on Apple Silicon (M1/M2/M3/M4) and Python 3.10+.

Quick Start

Python API

from mlx_meralion import load_model, transcribe

# Load model (auto-downloads from HuggingFace on first use)
model = load_model("MERaLiON/MERaLiON-2-10B-MLX")  # 10B 8-bit, recommended
# model = load_model("MERaLiON/MERaLiON-2-3B-MLX")   # 3B fp16, smaller

# Transcribe speech
text = transcribe(model, "audio.wav")
print(text)

# Translate to Chinese
text = transcribe(model, "audio.wav", task="translate_zh")

# Spoken question answering
text = transcribe(model, "audio.wav", task="sqa", question="What is the speaker talking about?")

Batch Inference

Process multiple audio files with GPU-batched decoding for higher throughput:

from mlx_meralion import load_model, batch_transcribe

model = load_model("MERaLiON/MERaLiON-2-10B-MLX")

results = batch_transcribe(model, ["a.wav", "b.wav", "c.wav", "d.wav"])
for text in results:
    print(text)

CLI

mlx-meralion --model MERaLiON/MERaLiON-2-10B-MLX --audio audio.wav --task asr
mlx-meralion --model MERaLiON/MERaLiON-2-10B-MLX --audio audio.wav --task translate_zh

Supported Tasks

Available Models

Batch Inference Benchmark

Measured on Apple M4 Pro (24 GB) with MERaLiON-2-10B-MLX, 8 audio samples (~25s each), max 256 tokens. Correctness validated by comparing batch outputs against sequential outputs token-for-token.

Features

• Apple Silicon native: Runs entirely on MLX with GPU acceleration
• Batch inference: GPU-batched decoding via BatchKVCache for multi-audio throughput
• N-gram blocking: Prevents repetitive output (matching HuggingFace quality)
• Smart chunking: Long audio split at 30s boundaries; short tails merged to prevent hallucination
• Auto-download: Models downloaded and cached from HuggingFace automatically

Architecture

Audio (WAV/MP3/FLAC)
  -> Whisper Encoder (1280-d)
    -> LayerNorm + MLP Adaptor
      -> Speech embeddings merged into text sequence
        -> Gemma2 Decoder -> text output