mlx-optiq

Run any LLM locally on your Mac. Quantize it, serve it, and code with it.

Website: https://mlx-optiq.com | Docs: https://mlx-optiq.com/docs/ | Models: https://mlx-optiq.com/models | Blog: https://mlx-optiq.com/blog/ | HF org: https://huggingface.co/mlx-community

mlx-optiq is the local-LLM stack for Apple Silicon: an optimizing compiler and runtime for MLX that turns a full-precision model into the best version for a given memory and latency budget on your Mac, using per-layer sensitivity measurement instead of uniform 4-bit everywhere. The same signal drives weights, KV cache, LoRA fine-tuning, and runtime adapter swapping.

One pip install gives you three ways to work with a local model: the CLI (quantize, serve, fine-tune), OptiQ Lab (a local web workbench), and OptiQ Code (a terminal coding agent that drives your served model).

pip install mlx-optiq

Python 3.11+. Quantizing and local inference need Apple Silicon; optiq code and optiq lab also run on Linux and Windows against any OpenAI-compatible base_url.

What it does

Mixed-precision weight quantization that beats uniform 4-bit at the same size. optiq convert measures each layer's sensitivity and allocates bits per layer. A static method assigns bits by architecture for models too large to measure. Methods.
SSD expert streaming runs large MoE quants that don't fit in RAM. A 2-bit Qwen3.5-122B-A10B runs on a 36 GB Mac at ~12 GB resident, experts streamed off disk. How.
Mixed-precision KV cache for longer context at lower memory. optiq serve runs a per-layer KV quant pipeline.
One server, two protocols. optiq serve speaks both the OpenAI and Anthropic APIs from one process. Point Claude Code or either SDK at the same local URL.
Speculative decoding via bundled MTP heads or paired drafters (--mtp, --drafter).
Distributed inference across Macs. optiq cluster serve shards a model's layers across two or more Macs over Thunderbolt and exposes one OpenAI endpoint. A 2-bit Qwen3.5-122B-A10B (42.8 GiB) runs fully resident across a 36 GB + 24 GB pair at ~20 tok/s, against 4.9 tok/s streaming experts off SSD on one Mac. How.
Sensitivity-aware LoRA (SFT + DPO) and runtime hot-swap adapters.
OptiQ Lab (pip install mlx-optiq then optiq lab): a local web UI for chat, quantize, fine-tune, and dataset work. Product.
OptiQ Code (optiq code): a terminal coding agent that drives whatever optiq serve is serving, offline, engineered for local models (never an empty patch, edit-resilient, stall-proof). Product.

Quickstart

Every mlx-optiq quant loads with stock mlx-lm:

from mlx_lm import load, generate
model, tok = load("mlx-community/Qwen3.5-9B-OptiQ-4bit")
print(generate(model, tok, prompt="Hello", max_tokens=50))

Installing mlx-optiq unlocks the rest. A few starting points:

# Serve with the OpenAI + Anthropic API and ~1.4x speculative decode
optiq serve --model mlx-community/Qwen3.5-9B-OptiQ-4bit --mtp

# Run a huge MoE that doesn't fit in RAM (experts stream off SSD)
optiq serve --model mlx-community/Qwen3.5-122B-A10B-OptiQ-2bit --stream-experts

# Quantize a fresh model (exact sensitivity, or fast structural rules for big bases)
optiq convert Qwen/Qwen3.5-9B --target-bpw 5.0 --candidate-bits 4,8
optiq convert <large-moe> --method static --candidate-bits 2,4 --target-bpw 2.5

# Fine-tune with sensitivity-aware LoRA
optiq lora train mlx-community/Qwen3.5-9B-OptiQ-4bit --data ./jsonl_dir --rank 8

# Shard a model across two Macs over Thunderbolt (one OpenAI endpoint)
optiq cluster up                                    # on every Mac
optiq cluster serve --model mlx-community/Qwen3.5-122B-A10B-OptiQ-2bit

# Code with your local model, in your terminal (offline)
optiq code                                          # interactive, in a repo
optiq code -p "Fix the failing test in parser.py"   # headless

Full guides for serving, KV-quant, LoRA, MTP and per-family setup are in the docs. The models page lists every quant with its Capability Score, and the blog has the deeper write-ups.

Requirements

Apple Silicon (M1 or newer), macOS, Python 3.11+.
The published quants load with stock mlx-lm. Converting and some MoE / multimodal runtime features track mlx-lm main; install it from git when a model card asks for it.

License

MIT for the package. Quantized models follow their base model's license.

Release files for mlx-optiq 0.4.29

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mlx-optiq 0.4.29
File	Size	Uploaded
mlx_optiq-0.4.29.tar.gz	2.2 MB	Aug 26, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for mlx-optiq 0.4.29
File	Interpreter	ABI	Platform	Reset
mlx_optiq-0.4.29-py3-none-any.whl 2.1 MB Aug 26, 2026	Python 3	none	any	Details

Total release size: 4.3 MB

Release files / mlx_optiq-0.4.29.tar.gz

Download URL	mlx_optiq-0.4.29.tar.gz
Size	2.2 MB
Tags	Source
SHA-256 checksum How to use checksums	`70421b8c500445d78db04c710add9ffff750c44044e474c14c814b9d055b83d6`
BLAKE2b-256 checksum How to use checksums	`bfa3083851345b3a5bdefeea0379cc3611d9a71c99ec848c3f6363d1c3741cd3`
Upload date	Aug 26, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.13.7`

Release files / mlx_optiq-0.4.29-py3-none-any.whl

Download URL	mlx_optiq-0.4.29-py3-none-any.whl
Size	2.1 MB
Tags	Python 3
SHA-256 checksum How to use checksums	`83a95556bbe5d22e56502a21a42ef75dd506b3f6bdc6f612c2020431c0ae46f6`
BLAKE2b-256 checksum How to use checksums	`c33dcd2a0fc1c0e25e402864d51e4dfd6cb166bf181154e9a7185cc6bda872fb`
Upload date	Aug 26, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.13.7`