gigachat-audio-mlx 0.1.2

Native GigaChat Audio inference on Apple Silicon with MLX

GigaChat Audio MLX

English · Русский

Local, offline audio understanding on Apple Silicon Macs. Ask questions about audio, transcribe or translate speech, detect emotion, find moments by time, and create timestamped summaries. This is an independent native MLX port of the official GigaChat3.1-Audio-10B-A1.8B model from the GigaChat Audio paper. It runs without a PyTorch, CUDA, or cloud runtime.

PyPI · Hugging Face models · Latest release · Full results · Upstream model · Original paper

At a glance: what each model needs

Variant	Model size	Free disk to set aside	Measured total process memory	5:17 audio on M4 Pro	Recommendation
BF16	22.54 GB	about 25 GB	22.84 GiB	10.41 s · 30.5× realtime	default for quality
Q8/BF16	13.75 GB	about 16 GB	14.80 GiB	6.90 s · 46.0× realtime	experimental opt-in

In plain terms: on the tested Mac, a 5-minute 17-second recording took about 10 seconds with BF16 or 7 seconds with Q8/BF16. Download time is not included. The free-disk figures are practical rounded reserves; download --dry-run reports the exact requirement before transferring anything.

The memory column is measured use by the whole process, not a promise that a Mac with exactly that much unified memory will work. Both variants are verified only on a 48 GB Mac today. 16/24/32 GB support will be stated only after it is tested.

Quick start

The verified environment is an Apple M4 Pro with 48 GB unified memory, macOS 15.7.7, and native ARM Python 3.12 or 3.13. Broader M1–M5 and lower-memory support is not yet claimed.

brew install uv
uv tool install gigachat-audio-mlx

gigachat-audio-mlx doctor
gigachat-audio-mlx download --variant bf16 --dry-run
gigachat-audio-mlx download --variant bf16 --yes

gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768

doctor is read-only. The download dry run shows exact bytes, cache hits, free disk space, and the destination before any network transfer.

Why this port?

• Native MLX inference without PyTorch, CUDA, ONNX Runtime, Core ML, or a cloud API.
• One local runtime for audio QA, transcription, translation, emotion analysis, temporal grounding, and timestamped summaries.
• Data-only model repositories: no remote Python execution and no trust_remote_code.
• Immutable model revisions, a signed manifest of expected files, and SHA-256 validation before loading.
• A quality-first BF16 model and a clearly labelled, optional Q8/BF16 variant.

Model variants and quality trade-off

All resource figures below are direct measurements on the same 317.27-second audio workload. Lower is better. The median of five clean processes is reported for time and whole-process physical memory.

Variant	Recommended use	Model size	Process memory	5-minute time	Hugging Face
BF16	recommended quality-first default	22.54 GB	22.84 GiB	10.41 s	ai-babai/gigachat-audio-mlx
Q8/BF16	experimental opt-in	13.75 GB	14.80 GiB	6.90 s	ai-babai/...-q8-bf16

Q8/BF16 is 39.0% smaller, used 35.2% less process memory, and was 33.7% faster. It also produced a factual error in 29 of 64 reviewed responses versus 26 of 64 for BF16. BF16 therefore remains the default.

Common tasks

The text instruction is part of every input. Use a preset or provide an exact custom request:

# Russian transcription
gigachat-audio-mlx run speech.wav --variant bf16 --task transcribe

# Translate speech to English
gigachat-audio-mlx run interview.flac --variant bf16 --task translate-en

# Find a moment in time
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --prompt "Find when the budget decision was approved. Return start and end timestamps."

# Timestamped summary
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768 \
  --stats

Available presets are transcribe, translate, translate-ru, translate-en, emotion, ground, summarize, and qa. A custom --prompt overrides the preset.

WAV and FLAC work in the default installation. MP3 is supported by the bundled audio wheel; M4A uses the macOS-native decoder fallback. Stereo is mixed to mono and resampled deterministically to 16 kHz with SoXR HQ. No Homebrew or ffmpeg installation is required.

Results available today

MLX BF16 compared with MLX Q8/BF16

Metric, lower is better	MLX BF16 · recommended	MLX Q8/BF16 · experimental	Change
Model size, decimal GB	22.54	13.75	−39.0%
Whole-process memory, 5-minute profile	22.84 GiB	14.80 GiB	−35.2%
End-to-end time, 5-minute profile	10.41 s	6.90 s	−33.7%
Responses with a factual error	26/64 (40.63%)	29/64 (45.31%)	+3 responses (+4.69 pp)

Model size is the disk payload, not RAM. Memory is the whole-process macOS physical footprint. Published performance figures come from the verified M4 Pro / 48 GB machine and are not universal Apple Silicon claims.

Original PyTorch BF16 compared with the MLX port

Check	Original PyTorch BF16	MLX BF16	MLX Q8/BF16
Model size, decimal GB	23.73	22.54	13.75
Exact decoded-text agreement	Reference	41/50 vs Original	45/50 vs MLX BF16

Exact text agreement is an implementation-parity diagnostic, not an aggregate quality score. Original PyTorch has not yet been measured with the same five-minute Mac resource protocol, so it is intentionally absent from the RAM and speed comparison. That measurement is planned after publication.

Results reported by the original authors

These values come from the upstream model card and were not rerun by this project.

Task and dataset	Metric	Original reported result
Audio QA · MMAU	Accuracy ↑	62.2
Audio QA (Russian) · RuBQ	Accuracy ↑	60.0
Emotion · Dusha podcast	Accuracy ↑	92.4
ASR (Russian) · FLEURS ru	WER ↓	4.4
ASR (English) · FLEURS en	WER ↓	6.5
Translation · FLEURS ru→en	BLEU ↑	33.4
Translation · FLEURS en→ru	BLEU ↑	26.0
Temporal localization · 20–60 min	mIoU ↑	48.3
Timed summary · 20–60 min	Overall ↑	67.9

See the full result tables and measurement notes and the upstream model card.

Python API

from gigachat_audio_mlx.runtime import GigaChatAudioRuntime

runtime = GigaChatAudioRuntime.load("/path/to/mlx-model")

Pinned Hugging Face snapshots can also be loaded without remote code:

runtime = GigaChatAudioRuntime.from_pretrained(
    "ai-babai/gigachat-audio-mlx",
    revision="db430193908ea810d4873404b43a148d5cbe9753",
    local_files_only=True,
)

Compatibility and limitations

• Verified: Apple M4 Pro, 48 GB unified memory, macOS 15.7.7 arm64, and Python 3.12/3.13.
• Not yet claimed: all M1–M5 Macs, 16/24/32 GB configurations, Intel Mac, Linux, Windows, iOS, or 90-minute inputs.
• Long timestamped summaries can reach the output-token limit. Increase --max-tokens only after checking the memory estimate.
• Answers, translations, emotions, facts, and timestamps can be wrong. Verify important output against the source audio.
• Do not use output as the sole basis for medical, legal, employment, safety, or surveillance decisions. Users remain responsible for consent, privacy, copyright, and the terms governing their audio.

Reproducibility and development

The source model is pinned to revision bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb. Model weights, datasets, caches, private audio, and large raw benchmark outputs are not stored in Git. The repository contains compact results, manifests, selection rules, and commands needed to inspect the published evidence.

Start with benchmarks/README.md for the benchmark map and docs/release/v0.1.0/RESULTS.md for the current tables and limitations.

License and attribution

The MLX runtime is MIT-licensed. Upstream model, paper, code, and dependency provenance is recorded in THIRD_PARTY_NOTICES.md. Please cite both this software through CITATION.cff and the original work:

• GigaChat3.1-Audio-10B-A1.8B model
• GigaChat Audio: Time-aware Large Audio Language Model, by Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin
• Upstream MIT license declaration

This project is not an official release by the GigaChat Audio authors. Security reports and contributions are covered by SECURITY.md and CONTRIBUTING.md.