mmh3turbo

MiniMax-H3 video + audio generation on Apple Silicon, with hand-written int8 Metal kernels.

uvx mmh3turbo "a rainbow colored skunk leaps over a mossy log in a supermarket"

That is the whole setup. The first run downloads what it needs and starts generating; later runs begin immediately. Prepared bundles come from yunfengwang/mmh3turbo-bundles; set MMH3TURBO_BUNDLE_REPO="" to convert them locally instead (~8 min). Output is an mp4 with a stereo soundtrack, plus the individual frames and a gif preview.

What this is

H3 is not a language model. It is a 33B diffusion transformer that denoises video and audio latents jointly in one packed sequence, conditioned by a frozen Qwen3-VL-32B text tower, with separate video and audio VAEs. Attention is full self-attention over [text | keyframes | audio | video] — no cross-attention, no per-modality weights.

This is a from-scratch MLX port. Every component is checked against a PyTorch transcription of the reference:

Why it is fast, and where it is not

The four fused Metal kernels are the point. flash_i8 is 26% faster than mx.fast.scaled_dot_product_attention at T=38656 (22.4 vs 16.6 TOPS), and the block's four GEMMs run at 44-50 TOPS.

Measured on an M5 Pro (51 GB), 12 steps:

A 1376x768 / 5 s / 15-step run takes 43 min here, against ~45 min reported for an M5 Max — a machine with roughly twice the GPU cores.

That is the floor, not a missing optimization. Attention is 74% of the block at 768p and scales with T². Eight approaches were measured and rejected: tile sweeps, K/V reuse, removing the score staging buffer, MLX's own SDPA, per-block eval, schedule reshaping, step caching, and temporal chunking. Each is documented with its numbers in kernels.py and the relevant module. The binding constraint is that QK's K is head_dim=128, and int8 GEMM manages 3.3 TOPS at K=128 versus 49.9 at K=5376 — so the peak figure is simply unreachable for this shape.

Want it faster? Use a smaller canvas. 352p is 1.5 min for 2 s.

Downloads: ~33 GB, not ~115 GB

The text tower is consumed as a 7.9 GB GGUF instead of the 66.7 GB unquantized encoder, which required implementing k-quant dequantization directly — and finding a real Q3_K bug along the way (ggml's high-mask bit advances across both half-blocks; resetting it per half-block is silently wrong, and the weight statistics look perfectly healthy either way).

Usage

mmh3turbo "prompt" -r 480p --seconds 5.0 -o out/
mmh3turbo --list-res
mmh3turbo "prompt" --first-frame photo.png     # image-to-video

Canvas dimensions must be multiples of 32 (the VAE's 16x times the DiT's 2x patch). 1280x720 is rejected with the nearest valid size, because otherwise the patch grid silently drops the last latent row and quietly returns 1280x704.

--steps defaults to 12; it is visually indistinguishable from the reference template's 20 (31.0 dB), while 8 gives 23.6 dB and 4 gives 17.6 dB.

Requirements

Apple Silicon, macOS. ffmpeg for the mp4 mux (frames and audio.wav are still written without it). Peak memory is 22 GiB at 352p and 31 GiB at 768p.

Source

https://github.com/vra/mmh3turbo

Credit

The base int8 GEMM and flash-attention kernels in _metal.py are vendored from the zimgturbo project. The reference semantics were read from ComfyUI's MiniMax-H3 implementation.

License

MIT