roocode-code-indexer-macos

High-performance OpenAI-compatible embedding server for Roo Code codebase indexing on Apple Silicon.

Runs Qwen3 embedding models locally via MLX with optimized batched GPU inference — no API keys needed. Up to 5x faster than Ollama for the same models.

Quick Start

# From PyPI
uvx roocode-code-indexer-macos

# Or from GitHub (latest)
uvx --from git+https://github.com/dmarkey/roocode-code-indexer-macos roocode-code-indexer-macos

Or with the recommended 4B model:

uvx roocode-code-indexer-macos --model mlx-community/Qwen3-Embedding-4B-4bit-DWQ

Roo Code Configuration

In Roo Code settings, set the embedding provider to OpenAI-compatible and point it at:

http://localhost:8000/v1

Example using mlx-community/Qwen3-Embedding-8B-4bit-DWQ (4096 dimensions). mlx-community/Qwen3-Embedding-4B-4bit-DWQ (2560 dimensions) is recommended for most use cases.

Available Models

Models are downloaded automatically from HuggingFace on first use.

Benchmarks: MLX vs Ollama

Synthetic code-like text data (10 concurrent requests x 60 texts = 600 texts), simulating Roo Code's indexing burst pattern. Reproducible via python benchmark.py.

Cold cache (first indexing run)

Warm cache (re-indexing with overlapping files)

Ollama has no embedding cache — every re-index pays full inference cost.

Why is MLX faster?

• Request coalescing — concurrent requests are merged into single GPU batches instead of queuing serially
• Length-bucketed sub-batching — minimizes padding waste across variable-length texts
• Compiled Metal kernels — mx.compile with fixed padding buckets for graph reuse
• LRU embedding cache — repeated texts skip inference entirely
• Last-token pooling — matches Qwen3-Embedding training objective

Environment Variables

Requirements

• macOS with Apple Silicon (M1/M2/M3/M4)
• Python 3.10+

License

MIT