MLX MCP Server

MCP server bridging Codex to local MLX LM (and any OpenAI-compatible backend)

AI automation consulting — I help businesses replace painful manual processes with LLM-powered pipelines and workflow automation.

Overview

mlx-mcp-server is a Model Context Protocol server that bridges Codex to a local LLM (MLX LM on Apple Silicon, or any OpenAI-compatible /v1 backend) so Codex can hand off eligible work to a free, private model instead of spending paid tokens.

Rather than a plain pass-through, it enforces an offload-first policy: its headline iterate tool runs a self-correcting escalation ladder — retry locally against a gate, step up to a bigger local model, and only hand back to Codex when the local rungs are exhausted. The result is lower token cost and more privacy for routine work (summarizing, extracting, classifying, boilerplate, single-file review), while Codex stays focused on multi-file reasoning and judgment calls.

Features

• Offload-first, token-thrift — routes eligible work (summarize, extract, classify, boilerplate, single-file review, first drafts) to a free, private local model before spending paid Codex tokens.
• Self-correcting iterate ladder — retries locally, then a bigger local model, then hands back to Codex; free rungs are exhausted before any paid work.
• Gated retries — structural gates (require_json / schema_keys / contains / regex / min_len) and an executable gate (run a linter or test against $CANDIDATE_FILE) let the local model fix its own output.
• Category profiles and telemetry — summarize, extract, explain, boilerplate, review, and other tune local retry behavior and log content-free savings data.
• Runtime model switching — swap the active model by name or fuzzy fragment with no Codex restart; choice persists across restarts.
• Work-hours guard — optionally blocks large (>22 GB RAM) models during work hours to avoid swap thrashing.
• One-step install — writes the Codex MCP entry to ~/.codex/config.toml by default, with legacy Claude Code/Desktop support still available.
• Backend-agnostic — works with MLX LM or any OpenAI-compatible /v1 endpoint; content-free usage metrics, MIT licensed, tested and CI'd.
• Built-in observability package — Prometheus exporter, Alloy pipeline, Terraform dashboards, and rollout docs for tracking offload ROI, routing discipline, and local inference health.

Getting Started

Requires Python 3.11+ and a running OpenAI-compatible LLM backend (MLX LM on Apple Silicon is recommended).

pip install mlx-mcp-server
mlx-mcp-server install   # wire into Codex, then restart Codex or check /mcp

See Usage for backend setup, configuration, and the full tool reference.

Usage

1. Start a local LLM backend

The server talks to any OpenAI-compatible /v1 endpoint. On Apple Silicon, MLX LM is recommended:

pip install mlx-lm
mlx_lm.server --model mlx-community/Qwen2.5-Coder-14B-Instruct-4bit   # serves on http://localhost:8080

Any other OpenAI-compatible backend works too (e.g. Ollama on Linux/Intel — point MLX_BASE_URL at http://localhost:11434).

2. Install and wire it into Codex

pip install mlx-mcp-server

# One-step setup for Codex user config
mlx-mcp-server install

# ...or project-local Codex config
mlx-mcp-server install --codex-project

# Preview the config without writing anything
mlx-mcp-server install --dry-run

The installer writes an mlx entry into Codex config (~/.codex/config.toml by default) with [mcp_servers.mlx], environment variables, and approval-mode entries for iterate, chat, quick_test, health_check, and list_models. Restart Codex or run /mcp, then call health_check to verify the backend is reachable. Run mlx-mcp-server help for the full CLI reference.

3. How Codex uses it — offload-first

The whole point is token thrift: Codex routes eligible work (summarize, extract, classify, reformat, boilerplate, single-file review, simple refactors, first drafts) to your free, private local model before spending paid tokens. The headline tool is iterate, which runs a self-correcting escalation ladder:

local model retries (feeding each gate failure back in) → a bigger local model → hand back to Codex

Free rungs are exhausted before any paid work happens. You attach a gate so the local model can self-correct:

• Structural gates (cheap, content-free): require_json, schema_keys, contains, regex, min_len
• Executable gate: check_command — a shell command that sees the candidate at $CANDIDATE_FILE and exits 0 to pass (e.g. a linter or test)

With no gate, iterate runs a single local attempt and asks Codex to verify.

# Generate boilerplate, gated by a linter — retries locally until ruff is happy
iterate(message="write a Python slugify() function",
        category="boilerplate",
        check_command="ruff check $CANDIDATE_FILE")

# Extract structured data, gated on valid JSON with required keys
iterate(message="extract name, email, company from this signature: ...",
        category="extract",
        require_json=true,
        schema_keys=["name", "email", "company"])

# Quick one-off to the local model, no iteration
chat(message="explain what this regex does: ^\\d{3}-\\d{4}$")

Tools

Codex can inspect the server with /mcp and overall session state with /status. The bundled /switch-model and /mlx-help slash commands remain available for legacy Claude Code installs.

Configuration

Set as env vars in the MCP server entry (the installer scaffolds these):

AGENTS.md offload policy

Codex reliably follows local offload when the project gives it explicit guidance. This repository includes AGENTS.md with the expected policy:

• call mcp__mlx__iterate before eligible local work
• call mcp__mlx__chat for quick one-shot summarize/explain/extract/review tasks
• always pass category as one of review, boilerplate, summarize, extract, explain, or other
• keep multi-file architecture, tool-heavy work, and final user-facing synthesis on Codex

Legacy Claude support

Claude Code and Claude Desktop config writers remain available for compatibility, but Codex is the primary target:

# Claude Code plus legacy slash commands
mlx-mcp-server install --claude-code --with-commands

# Claude Desktop
mlx-mcp-server install --claude-desktop

Observability

The observability/ package turns the server's content-free local telemetry into operator-grade dashboards:

• local Prometheus exporter over ~/.omlx/mlx-call-log.jsonl, ~/.omlx/hook-decisions.jsonl, and ~/.omlx/stats.json
• Alloy scrape and Grafana Cloud remote_write configuration
• Terraform-managed Grafana folder and dashboards
• docs for architecture, telemetry privacy, rollout gates, and PDF report artifacts

Phase 1 dashboards cover offload ROI, routing discipline, and local inference health. Phase 2 latency metrics are deliberately gated: the exporter only emits call duration histograms when --enable-call-latency is set after source-writer validation and log cutover.

Start here:

python3 -m unittest discover -s observability/exporter -p 'test_*.py'
python3 observability/exporter/mlx_mcp_exporter.py --once
terraform -chdir=observability/terraform validate

See observability/README.md for the architecture, privacy contract, rollout checklist, and Grafana/Terraform workflow.

Development

# Install dev dependencies
pip install -r requirements-dev.txt  # if present
# Run tests
pytest -q
# Run observability exporter tests
python3 -m unittest discover -s observability/exporter -p 'test_*.py'

License

MIT — see LICENSE.