GigaChat Audio MLX

English · Русский

Local, offline audio understanding on Apple Silicon Macs. Ask questions about audio, transcribe or translate speech, detect emotion, find moments by time, and create timestamped summaries. This is an independent native MLX port of the official GigaChat3.1-Audio-10B-A1.8B model from the GigaChat Audio paper. It runs without a PyTorch, CUDA, or cloud runtime.

PyPI · Hugging Face models · Latest release · Full results · Upstream model · Original paper

At a glance: what each model needs

In plain terms: on the tested Mac, a 5-minute 17-second recording took about 10 seconds with BF16 or 7 seconds with Q8/BF16. Download time is not included. The free-disk figures are practical rounded reserves; download --dry-run reports the exact requirement before transferring anything.

The memory column is measured use by the whole process, not a promise that a Mac with exactly that much unified memory will work. Both variants are verified only on a 48 GB Mac today. 16/24/32 GB support will be stated only after it is tested.

Why MLX matters on this Mac

Same Apple M4 Pro with 48 GB unified memory. The Original row is a safety stop, not a completed speed benchmark; the MLX rows are medians of five clean runs.

The pinned upstream Transformers implementation is demonstrated for CUDA. On MPS it exceeded the 48 GB Mac's safe memory budget and nearly filled system swap even on the short smoke. We stopped before the 5-minute run, so no Original-to-MLX speedup is claimed. The practical conclusion is simpler: MLX makes this model usable on the tested Mac.

Quick start

The verified environment is an Apple M4 Pro with 48 GB unified memory, macOS 15.7.7, and native ARM Python 3.12 or 3.13. Broader M1–M5 and lower-memory support is not yet claimed.

brew install uv
uv tool install gigachat-audio-mlx

gigachat-audio-mlx doctor
gigachat-audio-mlx download --variant bf16 --dry-run
gigachat-audio-mlx download --variant bf16 --yes

gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768

doctor is read-only. The download dry run shows exact bytes, cache hits, free disk space, and the destination before any network transfer.

Why this port?

• Native MLX inference without PyTorch, CUDA, ONNX Runtime, Core ML, or a cloud API.
• One local runtime for audio QA, transcription, translation, emotion analysis, temporal grounding, and timestamped summaries.
• Data-only model repositories: no remote Python execution and no trust_remote_code.
• Immutable model revisions, a signed manifest of expected files, and SHA-256 validation before loading.
• A quality-first BF16 model and a compact Q8/BF16 option for lower disk, memory, and processing time.

Choose between BF16 and Q8/BF16

All resource figures below are direct measurements on the same 317.27-second audio workload. Lower is better. The median of five clean processes is reported for time and whole-process physical memory.

Q8/BF16 is 39.0% smaller, used 35.2% less process memory, and needed 33.7% less processing time. In the blinded review, it had factual errors in 3 more cases than BF16. Choose Q8/BF16 when size, memory, and speed matter; choose BF16 for the best measured answer quality. The exact review counts are shown below.

Common tasks

The text instruction is part of every input. Use a preset or provide an exact custom request:

# Russian transcription
gigachat-audio-mlx run speech.wav --variant bf16 --task transcribe

# Translate speech to English
gigachat-audio-mlx run interview.flac --variant bf16 --task translate-en

# Find a moment in time
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --prompt "Find when the budget decision was approved. Return start and end timestamps."

# Timestamped summary
gigachat-audio-mlx run meeting.m4a \
  --variant bf16 \
  --task summarize \
  --max-tokens 768 \
  --stats

Available presets are transcribe, translate, translate-ru, translate-en, emotion, ground, summarize, and qa. A custom --prompt overrides the preset.

WAV and FLAC work in the default installation. MP3 is supported by the bundled audio wheel; M4A uses the macOS-native decoder fallback. Stereo is mixed to mono and resampled deterministically to 16 kHz with SoXR HQ. No Homebrew or ffmpeg installation is required.

Results available today

MLX BF16 compared with MLX Q8/BF16

Model size is the disk payload, not RAM. Memory is the whole-process macOS physical footprint. Published performance figures come from the verified M4 Pro / 48 GB machine and are not universal Apple Silicon claims.

Original PyTorch BF16 compared with the MLX port

Exact text agreement is an implementation-parity diagnostic, not an aggregate quality score.

The post-publication Original resource check did not produce a valid latency number on this Mac. The pinned official PyTorch/MPS path had not completed a 9.84-second smoke after 277 seconds; its process footprint had reached at least 51.92 GiB and system swap was nearly full. We stopped it before starting the 317.27-second profile. Therefore Original remains N/A for same-Mac time and memory rather than being compared with a failed or swap-dominated run.

Results reported by the original authors

These values come from the upstream model card and were not rerun by this project.

See the full result tables and measurement notes and the upstream model card.

Python API

from gigachat_audio_mlx.runtime import GigaChatAudioRuntime

runtime = GigaChatAudioRuntime.load("/path/to/mlx-model")

Pinned Hugging Face snapshots can also be loaded without remote code:

runtime = GigaChatAudioRuntime.from_pretrained(
    "ai-babai/gigachat-audio-mlx",
    revision="db430193908ea810d4873404b43a148d5cbe9753",
    local_files_only=True,
)

Compatibility and limitations

• Verified: Apple M4 Pro, 48 GB unified memory, macOS 15.7.7 arm64, and Python 3.12/3.13.
• Not yet claimed: all M1–M5 Macs, 16/24/32 GB configurations, Intel Mac, Linux, Windows, iOS, or 90-minute inputs.
• Long timestamped summaries can reach the output-token limit. Increase --max-tokens only after checking the memory estimate.
• Answers, translations, emotions, facts, and timestamps can be wrong. Verify important output against the source audio.
• Do not use output as the sole basis for medical, legal, employment, safety, or surveillance decisions. Users remain responsible for consent, privacy, copyright, and the terms governing their audio.

Reproducibility and development

The source model is pinned to revision bf73d03a43bdf5118f5a4dbdc24ba6f56ac31cfb. Model weights, datasets, caches, private audio, and large raw benchmark outputs are not stored in Git. The repository contains compact results, manifests, selection rules, and commands needed to inspect the published evidence.

Start with benchmarks/README.md for the benchmark map and docs/release/v0.1.0/RESULTS.md for the current tables and limitations.

License and attribution

The MLX runtime is MIT-licensed. Upstream model, paper, code, and dependency provenance is recorded in THIRD_PARTY_NOTICES.md. Please cite both this software through CITATION.cff and the original work:

• GigaChat3.1-Audio-10B-A1.8B model
• GigaChat Audio: Time-aware Large Audio Language Model, by Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov, Alexandr Maximenko, Oleg Kutuzov, Pavel Bogomolov, and Fyodor Minkin
• Upstream MIT license declaration

This project is not an official release by the GigaChat Audio authors. Security reports and contributions are covered by SECURITY.md and CONTRIBUTING.md.