rapid-mlx

The fastest local AI engine for Apple Silicon.
Drop-in OpenAI / Anthropic API · up to 3× Ollama's throughput (measured) · Runs on any M-series Mac.

_{rapidmlx.com ·
Docs ·
Model mirror ·
Desktop app}

Quick Start (60 seconds)

1. Install — pick one path (run only one of these):

One-liner — detects your RAM, picks a starter model (recommended):

curl -fsSL https://rapidmlx.com/install.sh | bash

or Homebrew — prebuilt bottle straight from homebrew-core:

brew install rapid-mlx

Both land the same rapid-mlx CLI. The curl installer additionally installs Python 3.10+ if missing, creates an isolated venv at ~/.rapid-mlx/, symlinks the rapid-mlx CLI into ~/.local/bin/, and prints a serve command sized to your Mac (8–15 GB → lfm2.5-2.6b-4bit; 16–17 GB → qwen3.5-4b-4bit; 18–23 GB → qwen3.5-9b-4bit; 24–31 GB → bonsai-27b-2bit; 32–63 GB → gemma-4-26b-4bit; 64–95 GB → qwen3.6-35b-8bit; 96 GB+ → qwen3.5-122b-mxfp4).

Install security. install.sh is served over HTTPS (HSTS-preload) from rapidmlx.com and is a byte-identical mirror of install.sh at the release commit — read it before running if you like. If you want a cryptographically verified installer rather than trusting the website pipe, don't curl | bash the URL above: instead download the release's install.sh asset, verify it against the cosign-signed SHA256SUMS.txt shipped alongside it, and run that verified copy — full recipe in SECURITY.md. PyPI artifacts additionally carry Sigstore attestations (PEP 740). Two more low-trust paths:

Pin to a commit hash — curl -fsSL https://raw.githubusercontent.com/raullenchai/Rapid-MLX/<commit>/install.sh -o install.sh && shasum -a 256 install.sh && bash install.sh

Skip the shell script entirely — use Homebrew, uv, or pip below.

See Alternative install methods for the non-curl paths.

2. Chat with a model right now:

rapid-mlx chat

Defaults to qwen3.5-4b-4bit. First run downloads the weights (~2.5 GB) with a progress bar and drops you into a REPL. Type /help for slash commands, /exit to quit.

3. Or serve it for use from other apps:

rapid-mlx serve qwen3.5-4b-4bit

Starts an OpenAI-compatible HTTP server bound to http://localhost:8000. Point any client that supports a local custom endpoint (Aider, LangChain, OpenCode, PydanticAI, your own scripts) at http://localhost:8000/v1; Claude Code / Anthropic SDK uses http://localhost:8000 (the Anthropic messages route lives at /v1/messages under the same host).

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}'

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
print(client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "Say hello"}],
).choices[0].message.content)

4. Or wire up your coding agent — one command:

rapid-mlx launch claude-code

With a server running (step 3), this patches Claude Code's local config (~/.config/claude/settings.json) to route at http://localhost:8000 — no manual env vars, no editing JSON by hand. You get a fully local Claude Code: $0 per token, nothing leaves your Mac. Swap in cline or continue-dev for the other IDE clients, or run rapid-mlx launch list to see what's detected on this machine.

Cursor: Cursor currently routes BYOK requests through its own servers, so its servers cannot reach a Rapid-MLX endpoint on localhost. Rapid-MLX therefore does not generate a Cursor localhost config. If you intentionally expose the server through a public HTTPS tunnel, set RAPID_MLX_API_KEY=your-secret for both rapid-mlx serve ... and rapid-mlx launch cursor --server-url https://your-public-host. This is no longer a fully local connection; never expose an unauthenticated server. Rapid-MLX rejects explicit local/private addresses but cannot verify reachability from Cursor's network, whose DNS view may differ from your Mac.

Vision / audio / video / diffusion models? Base install is text-only (~460 MB). Vision, audio (TTS, STT, voice cloning), video generation, embeddings, and DFlash speculative decoding ship as opt-in extras. → Optional extras

Not into the terminal? Rapid-MLX Desktop bundles the same engine inside a one-click Mac app.

Video generation

Run text-to-video or image-to-video locally through the OpenAI-compatible Videos API. Three backends ship — Wan 2.1 / 2.2, CogVideoX-Fun and LTX-2.3 — across 8 registered checkpoints. wan2.2-ti2v-5b-q8 is the recommended starting point: smallest of the Wan set, and TI2V means one checkpoint does both text-to-video and image-to-video.

Requires Python 3.11+ (the video runtime does not support 3.10; core text and audio still do) and ffmpeg for the final MP4 mux.

pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve wan2.2-ti2v-5b-q8

Create and download a clip:

curl http://localhost:8000/v1/videos \
  -F model=wan2.2-ti2v-5b-q8 \
  -F 'prompt=A fox running through fresh snow, cinematic tracking shot' \
  -F seconds=1 \
  -F size=832x512

# Poll until GET /v1/videos/VIDEO_ID reports "status": "completed", then:
curl http://localhost:8000/v1/videos/VIDEO_ID/content -o output.mp4

The create call returns a job immediately. Poll GET /v1/videos/VIDEO_ID until status is completed. Add -F input_reference=@start.png for image-to-video.

Generation is serialized — one clip at a time — because two diffusion pipelines resident at once will exhaust unified memory. Expect minutes of compute per second of footage, not real time.

→ Every checkpoint, RAM requirement and tuning knob

Audio: speech, transcription, voice cloning

41 audio aliases behind the OpenAI-compatible /v1/audio/* endpoints — any OpenAI SDK works unchanged.

pip install 'rapid-mlx[audio]'

# Text to speech
rapid-mlx serve kokoro
curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"kokoro","input":"hello from rapid-mlx"}' --output hello.wav

# Transcription (Whisper / Parakeet / SenseVoice)
rapid-mlx serve whisper-large-v3-turbo
curl http://localhost:8000/v1/audio/transcriptions \
  -F file=@hello.wav -F model=whisper-large-v3-turbo

Beyond the basics, three things you may not expect to run locally:

Zero-shot voice cloning from a reference clip. indextts is the only one that takes the clip alone; qwen3-tts-clone, f5-tts-zh and chatterbox all require ref_text (the clip's exact transcript) paired with ref_audio, and the request is rejected before generation if it is missing.
Voice design — qwen3-tts-voicedesign has no named speakers at all. Describe the voice you want in natural language via instructions (timbre, gender, age, accent, emotion, prosody) and it synthesises it.
Forced alignment — qwen3-aligner takes audio plus the transcript you already have and returns per-character timings. It never guesses at the words, so it cannot mis-hear them; that is what karaoke captions and beat-synced editing need.

Also: word-level timestamps on transcription, and local text-to-music at /v1/audio/music.

→ All 41 aliases across 12 families

Why Rapid-MLX


Apple-Silicon-native	Pure MLX kernels — no llama.cpp fallback, no Metal shim. Continuous batching, prompt cache (radix + DeltaNet RNN snapshots), and a quantized live KV cache (int4/int8 on the continuous-batching cache + TurboQuant K8V4 codec) run at native MLX bandwidth on M1 → M4.
Drop-in OpenAI / Anthropic API	`/v1/chat/completions`, `/v1/responses` (Codex CLI), `/v1/messages` (Anthropic SDK / Claude Code), `/v1/embeddings`, `/v1/audio/*`, `/v1/videos` — same wire as ChatGPT / Claude, no client adapter.
First-class ecosystem coverage	12 agent CLIs and 3 Python frameworks are wire-verified against real weights every release (5 are Tier-1, re-verified on current binaries) — Codex CLI, Claude Code, OpenCode, Qwen Code, OpenHands, Hermes Agent, Aider, Kilo Code, DeepSeek Harness, GitHub Copilot, Factory Droid, Moonshot Kimi Code + LangChain, PydanticAI, smolagents.

→ Full feature breakdown

Use Cases


Chat in the terminal	`rapid-mlx chat qwen3.5-9b-4bit`	Streaming REPL, `/help` for slash commands, `--think` / `--no-think` to control CoT.
OpenAI server for your apps	`rapid-mlx serve qwen3.5-9b-4bit`	Point Aider, LibreChat, Open WebUI, or LangChain at `http://localhost:8000/v1`.
Agent backends	`rapid-mlx serve qwen3.6-35b-8bit &` `rapid-mlx agents codex --setup && codex`	9 agents auto-configure via `agents <name> --setup` once the server is up (12 wire-verified total, 5 Tier-1) — see Agent support.
Benchmark your Mac	`rapid-mlx bench qwen3.5-9b-4bit --submit`	Standardized B=1 bench, opens a PR to publish your row on rapidmlx.com.

→ One-shot IDE setup with rapid-mlx launch <claude-code|cline|continue-dev>

Agent Support

All 12 agents below are wire-verified against real weights every release via their own integration-test cell. Of these, five are Tier-1 — Claude Code, Codex CLI, Hermes, Aider, and DeepSeek Harness — re-verified end-to-end against the current client binary every release, with one guardian per API wire (Anthropic /v1/messages, OpenAI /v1/responses, and /v1/chat/completions covered for tool-calling depth, reach, and DeepSeek's own harness protocol). The other seven are Tier-2: wire-verified in the matrix and configured on-demand. The first nine agents each ship a rapid-mlx agents <name> --setup config template (except Claude Code, which is one env-var); GitHub Copilot, Factory Droid, and Moonshot Kimi Code plug in through their own documented BYOK config (auth-gated, so the matrix cell is a wire smoke).

Tier-1 is not a label — it is a job that blocks the release. tests/integrations/agent_smoke.sh drives each of the five through the same real multi-step bug-fix task against a local 35B model and asserts the repo's own test suite goes green afterwards; if any one of them fails, the version cannot tag or publish.

Tier-1 (5): Claude Code · Codex CLI · Hermes · Aider — last re-verified end-to-end 2026-07-28 on current binaries (claude 2.1.211, codex 0.145.0, hermes 0.9.0, aider 0.86.2) against rapid-mlx 0.11.1. DeepSeek Harness — promoted 2026-08-17, verified on dsh 0.1.0-rc.7 against qwen3.6-35b-8bit. Tier-2 (7): OpenCode · Qwen Code · OpenHands · Kilo Code · GitHub Copilot · Factory Droid · Moonshot Kimi Code.

Agents (12)	Frameworks (3)
Codex CLI · Claude Code · OpenCode · Qwen Code · OpenHands · Hermes Agent · Aider · Kilo Code · DeepSeek Harness · GitHub Copilot · Factory Droid · Moonshot Kimi Code	LangChain (+ LangGraph) · PydanticAI · smolagents

Also compatible with OpenAI-compatible clients that allow direct local endpoints via http://localhost:8000/v1 — LibreChat, Open WebUI, and more plug in with a single URL change.

→ Full 12-agent + 3-framework matrix (test cells + xfail reasons) → Codex CLI · Claude Code · OpenCode · Qwen Code · OpenHands · Hermes · Aider · Kilo Code · DeepSeek Harness · Copilot · Droid · Kimi Code

Choose Your Model

The installer and desktop app use the same RAM-tier recommendation catalog. Run rapid-mlx recipe to see its Smart and Fast picks for this Mac (--max-ram 32 simulates another tier; --json is machine-readable). If you want to shop the full catalog: rapid-mlx models lists every alias, rapid-mlx info <alias> shows the per-alias profile (parser, MoE / hybrid flags, KV codec eligibility, speculative-decoding gates).

This table is the same one the desktop app's picker reads, and the installer prints the matching line for your Mac — a CI test parses both files and fails if they drift apart. Measured rows use the standard ~8K prompt peak of the complete rapid-mlx serve process tree on an M2 Pro 32 GB Mac mini.

RAM	Recommended	Peak RSS	One-shot
8–15 GB MacBook Air / base Mini	`lfm2.5-2.6b-4bit`	3.2 GB	`rapid-mlx serve lfm2.5-2.6b-4bit`
16–17 GB MacBook Air / Pro	`qwen3.5-4b-4bit`	5.8 GB	`rapid-mlx serve qwen3.5-4b-4bit`
18–23 GB MacBook Pro	`qwen3.5-9b-4bit`	8.7 GB	`rapid-mlx serve qwen3.5-9b-4bit`
24–31 GB Mac Mini / MacBook Pro	`bonsai-27b-2bit`	13.0 GB	`rapid-mlx serve bonsai-27b-2bit`
32–63 GB Mac Studio / high-spec Mini	`gemma-4-26b-4bit`	20.0 GB	`rapid-mlx serve gemma-4-26b-4bit --no-mllm --kv-cache-dtype bf16 --cache-memory-mb 512`
64–95 GB Mac Studio	`qwen3.6-35b-8bit`	37.7 GB	`rapid-mlx serve qwen3.6-35b-8bit`
96 GB+ Mac Studio / Pro	`qwen3.5-122b-mxfp4`	—	`rapid-mlx serve qwen3.5-122b-mxfp4`

The 32–63 GB flags are not optional: Gemma 4 26B ships a vision tower that tier has no memory for, and an uncapped KV budget claims the headroom the rest of your Mac needs.

→ Full RAM tier map + serve flags per tier → Every alias, quant, and family (166 text + 41 audio + 8 video aliases, 215 total) · interactive at models.rapidmlx.com

Alternative install methods

The two paths above cover most users — reach for these only if you already manage Python yourself.

Homebrew — Mac-native, one command, prebuilt bottle from homebrew/core

brew install rapid-mlx

Ships in homebrew-core since 0.10.12 — no tap, no trust prompt. Upgrade with brew upgrade rapid-mlx. If you previously installed from the legacy raullenchai/rapid-mlx tap, switch once: brew uninstall rapid-mlx && brew untap raullenchai/rapid-mlx && brew install rapid-mlx.

uv — isolated tool install, auto-manages Python

uv tool install rapid-mlx@latest

Don't have uv yet? curl -LsSf https://astral.sh/uv/install.sh | sh. Upgrade with uv tool upgrade rapid-mlx.

pip — requires Python 3.10+ (macOS ships 3.9)

python3.12 -m pip install rapid-mlx

If pip install rapid-mlx says "no matching distribution", your Python is too old. brew install python@3.12 first. Upgrade with pip install -U rapid-mlx.

For image-input / VLM models (Qwen-VL, true multimodal), install the vision extra: pip install 'rapid-mlx[vision]' — see Optional extras.

For the complete feature set — vision, chat, embeddings, and audio — install the [all] extra: pip install 'rapid-mlx[all]'. Audio alone is pip install 'rapid-mlx[audio]'; see Optional extras.

Command Reference

rapid-mlx --help                    # top-level command list
rapid-mlx <subcommand> --help       # per-subcommand flags

Covers chat, serve, share, agents (setup / test), bench, models, pull, rm, ps, info, doctor, upgrade, telemetry, and launch.

→ Full CLI reference with every flag

Troubleshooting

Run the built-in self-check first:

rapid-mlx doctor

Top three things that go wrong:

Much slower than expected. Qwen3.5 / 3.6 default to thinking-on — add --no-think to skip chain-of-thought. → Slow tok/s
Out of memory. Model too big for your RAM — pick a smaller quant from Choose Your Model or the full tier map. → OOM guide
Tool calls arriving as plain text. Auto-recovery handles most cases; if not, set --tool-call-parser explicitly for your model. → Tool-call recovery

→ All troubleshooting entries (OOM, empty responses, slow TTFT, port taken, shell completion, HF cache, and more)

See it in action

Rapid-MLX demo — install, serve Gemma 4, chat, tool calling

Community & Contributing

Report a bug or request a model: Issues
Report a security issue: Private advisory — see SECURITY.md
Ask a question or share a build: Discussions
Contribute code, aliases, or docs: CONTRIBUTING.md
Add your hardware to the public benchmark: rapid-mlx bench <alias> --submit opens the PR for you

Rapid-MLX ships opt-in anonymous telemetry (off by default; explicit rapid-mlx telemetry enable required). No prompts, completions, paths, IPs, or API keys are ever collected. → What we do and don't collect

🚀 Contributors

Every avatar here shipped something in rapid-mlx — model support, tool-call parsers, fixes, docs, and benchmark submissions. Thank you.

Star History

License

Apache 2.0 — see LICENSE.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rapid_mlx-0.12.15.tar.gz (5.6 MB view details)

Uploaded Aug 18, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

rapid_mlx-0.12.15-py3-none-any.whl (2.8 MB view details)

Uploaded Aug 18, 2026 Python 3

File details

Details for the file rapid_mlx-0.12.15.tar.gz.

File metadata

Download URL: rapid_mlx-0.12.15.tar.gz
Upload date: Aug 18, 2026
Size: 5.6 MB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for rapid_mlx-0.12.15.tar.gz
Algorithm	Hash digest
SHA256	`d50901621414c15ab0b45b5895ee98e2768db6ae2d1c42eb34beefcc4398af28`
MD5	`e6623cca39d7903a4fab745f42f9d8ea`
BLAKE2b-256	`98127d812cc002d779640e3c7cba8fb6927529b24f7c9027eec66662a35717d3`

See more details on using hashes here.

Provenance

The following attestation bundles were made for rapid_mlx-0.12.15.tar.gz:

Publisher: publish.yml on raullenchai/Rapid-MLX

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: rapid_mlx-0.12.15.tar.gz
- Subject digest: d50901621414c15ab0b45b5895ee98e2768db6ae2d1c42eb34beefcc4398af28
- Sigstore transparency entry: 2501089022
- Sigstore integration time: Aug 18, 2026
Source repository:
- Permalink: raullenchai/Rapid-MLX@8d63b0c949bc19ffaf4c48379e9619f70c24d5bf
- Branch / Tag: refs/tags/v0.12.15
- Owner: https://github.com/raullenchai
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@8d63b0c949bc19ffaf4c48379e9619f70c24d5bf
- Trigger Event: release