gamemlx

GAME — MLX edition. Pure MLX inference for GAME (Guided Automatic MIDI Extraction): singing vocals → MIDI, running natively on Apple Silicon GPU.

• Numerically aligned with the official PyTorch implementation: on a 20-file test set, fp32 output matches torch CPU exactly — all onset/offset/pitch diffs 0.0000 (docs/ALIGNMENT.md)
• ~9× faster than torch CPU on M4 Air (0.172s vs 1.609s per file average), ~15–20× realtime
• Low memory: 111MB wired (fp16) / 206MB (fp32) on the medium model
• Automatic silence slicing → per-chunk inference → timeline-merged MIDI output (same behavior as upstream)
• Models auto-download from HuggingFace (ModelScope fallback)

Install

pip install gamemlx          # or: uv tool install gamemlx

Requires Apple Silicon (M1–M4) and Python ≥ 3.10.

Quick start

# alias auto-downloads weights (small | medium | large) on first use
gamemlx extract vocals.wav -m medium --language zh

# local model dir + options
gamemlx extract song/ -m ./weights/GAME-1.0-medium-mlx \
    --precision fp16 --language zh --tempo 120 \
    --seg-threshold 0.2 --seg-radius 0.02 --est-threshold 0.2 \
    --output-dir out/

Python API:

import gamemlx

model, lang_map = gamemlx.load_model("medium")          # → GAMEModel + {"en":1,"ja":2,"yue":3,"zh":4}
gamemlx.extract(model, "vocals.wav", language="zh")      # writes .mid next to the input

Supported languages: en, ja, yue, zh (improves segmentation; omit for language-agnostic).

Models

Converted from openvpi/GAME v1.0 checkpoints (scripts/convert_to_mlx.py). Cache: ~/.cache/gamemlx (override with GAMEMLX_CACHE).

Precision & performance

Apple Silicon GPUs execute fp16 and fp32 at the same rate; fp16's benefit here is memory, not speed. Speed optimizations (operator fusion via mx.compile, fused QKV, mask fast paths) are on by default — disable with GAMEMLX_OPT=0 for the bit-exact eager path. Details: docs/OPTIMIZATIONS.md, docs/BENCH.md.

Development

uv venv && uv pip install -e . mlx numpy librosa soundfile mido pyyaml safetensors huggingface_hub click

# convert weights from a GAME torch checkpoint
python scripts/convert_to_mlx.py --input /path/to/GAME-1.0-medium/model.pt --output weights/GAME-1.0-medium-mlx

# alignment test vs torch CPU (needs GAME repo + torch installed; see script headers for env vars)
python scripts/compare.py
python scripts/bench.py

Credits

• openvpi/GAME — the original model, training code and pretrained checkpoints (MIT). This project is an independent MLX inference port.
• Model weights follow the upstream license; conversion script included in this repo.

License

MIT — see LICENSE.