MLXSmith

Fine-tune language models on Apple Silicon. SFT, preference optimization, reinforcement learning, distillation, and serving — all native to MLX.

Status: Alpha (v0.1.9) · Validated on Qwen3-4B and Qwen3-1.7B

Features

• Supervised fine-tuning — LoRA and QLoRA with configurable optimizers
• Preference optimization — DPO, ORPO, IPO, CPO, SimPO, and more
• Reinforcement learning — GRPO with verifier-based rewards
• Knowledge distillation — Offline and online preference distillation
• KTO — Kahneman-Tversky Optimization from binary feedback
• Online DPO — Live preference tuning with LLM judge scoring
• Self-verification training — Policy gradient from self-assessed rewards
• Synthetic data generation — Generate, evolve, and filter training data
• External model backends — Use Codex, Claude, Gemini CLIs or any OpenAI-compatible API for data generation and judging
• Recursive training — Self-improving RLM loop with task generation and gating
• Serving — OpenAI-compatible API with streaming
• Web dashboard (Next.js) — Models, adapters, training, eval, chat, and serving UI
• Environment plugins — Reusable task and verifier packages for RL training
• Experimental mHC adapters — Optional block-local mHC patching for MLX transformer blocks (not a speedup)

Requirements

• macOS with Apple Silicon (M1 or later)
• Python 3.10+

Data tools, configuration, and project scaffolding work on any platform.

Install

pip install "mlxsmith[all]"

# Core only (data tools, config, scaffolding)
pip install mlxsmith

# Apple Silicon training
pip install "mlxsmith[mlx,llm]"

# Training + serving
pip install "mlxsmith[mlx,llm,serve]"

Quickstart

# 1. Create a project
mlxsmith init myproj && cd myproj

# 2. Verify your environment
mlxsmith doctor

# 3. Pull a model
mlxsmith pull mlx-community/Qwen3-4B-Instruct-2507-4bit

# 4. Pull training data
mlxsmith data pull --preset alpaca

# 5. Fine-tune
mlxsmith sft \
  --model cache/mlx/mlx-community__Qwen3-4B-Instruct-2507-4bit \
  --data data/sft

# 6. Serve the result
mlxsmith serve --model runs/sft_0001/adapter --port 8080

See Getting Started for a complete walkthrough.

End-to-end Smoke (Qwen3-1.7B)

This repo includes an end-to-end smoke run that validates the full pipeline (SFT → Pref → RFT → RLM) on Qwen/Qwen3-1.7B-MLX-4bit.

mlxsmith pull Qwen/Qwen3-1.7B-MLX-4bit
./scripts/exp_qwen3_1.7b_mlx_4bit_e2e_smoke.sh

# Optional: also smoke-test `mlxsmith serve` + OpenAI-compatible endpoint
SMOKE_SERVE=1 ./scripts/exp_qwen3_1.7b_mlx_4bit_e2e_smoke.sh

The smoke run uses qwen3_1.7b_mlx_4bit_smoke.yaml and the tiny datasets in data/sft and data/prefs.

Repo SFT (Qwen3-1.7B)

To build a small “MLXSmith repo assistant” adapter on top of Qwen/Qwen3-1.7B-MLX-4bit, use the repo-grounded SFT script:

# 1) Generate seed prompts from the repo
python3 scripts/make_repo_seed_prompts.py --out data/mlxsmith_prompts.jsonl

# 2) Generate responses (via Codex) + train LoRA
NUM=300 BATCH=4 ITERS=2000 LR=2e-4 ./scripts/exp_qwen3_1.7b_mlx_4bit_repo_sft.sh

Notes:

• The script uses codex exec by default. Override with MLXSMITH_CLI_CODEX_CMD if needed.
• Qwen output sanitization is enabled in the included configs via infer.strip_think: true.

Web Dashboard (Optional)

Run the API server, then start the Next.js dashboard:

# Terminal 1: start the OpenAI-compatible API
mlxsmith serve --model cache/mlx/mlx-community__Qwen3-4B-Instruct-2507-4bit --port 8080

# Terminal 2: start the dashboard
cd apps/web
npm install
npm run dev

The dashboard defaults to http://localhost:8080 for the API base URL (change in Settings if needed).

Training Modes

See Concepts for an explanation of each training mode.

Tools

External Model Backends

MLXSmith can use powerful cloud models for synthetic data generation and judging while keeping fine-tuning local on Apple Silicon.

Supported backends:

• cli — shell out to Codex/Claude/Gemini CLIs (or any command you provide)
• openai — call any OpenAI-compatible Chat Completions endpoint

Note: training commands (sft, pref, rft, rlm loop) still require a local training backend like mlx-lm.

CLI Backend — Shell out to Codex, Claude, or Gemini CLIs:

# Use a CLI model for prompt generation
export MLXSMITH__MODEL__BACKEND=cli
export MLXSMITH_CLI_CODEX_CMD='codex exec --full-auto --model gpt-5.2'

# If your CLI expects the prompt as an argument instead of stdin:
# export MLXSMITH_CLI_PROMPT_FLAG='--prompt'

mlxsmith synthetic prompts \
  --model codex \
  --seed-prompts data/seeds.jsonl \
  --num 100 \
  --out data/prompts.jsonl

# Use a CLI model as judge for filtering
mlxsmith synthetic sft \
  --model codex \
  --judge-backend cli \
  --judge-model claude \
  --prompts data/prompts.jsonl \
  --out data/sft.jsonl

OpenAI Backend — Use any OpenAI-compatible API:

export MLXSMITH__MODEL__BACKEND=openai
export OPENAI_API_KEY="sk-..."
export MLXSMITH_API_BASE="https://api.openai.com/v1"  # or any compatible endpoint

mlxsmith synthetic prompts \
  --model gpt-4o \
  --out data/prompts.jsonl

This enables cloud-quality data generation with local training — use frontier models to create and filter training data, then fine-tune efficiently on your Mac.

Documentation

License

MIT