💡 The Case in One Line

🏛️ Four-Tier Architecture

mμ|LLM routes every query through four escalating tiers, short-circuiting the moment an answer is found. Cost accumulates only if cheaper tiers can't satisfy the query.

The classifier — DeBERTa-v3-small (44 MB) — reads the query once and predicts tier + intent category before any LLM is invoked. Tier 0 overhead: <1 ms.

Software Factory: from specification to evidence

Build small, dependent tasks with local inference, protected executable tests, bounded repair loops, and a human handoff when the worker needs help. Inspect the work on a flat operations board, or switch on the animated 3D factory.

How it works · Watch the tour · Spec → working game: evidence · Try Peg Bloom

Build several projects together and see which need a human: three marketing workspaces, their contracts, repairs, and review evidence. Executable checks establish technical behavior; a separate product gate asks whether the result is useful and usable.

The Peg Bloom case study includes model-generated source, independent checks for three puzzle boards and 63 legal jumps, browser validation, failed attempts, diagnostic guidance, and an artifact hash. An accepted artifact means its configured checks passed; the evidence makes that claim inspectable.

🚀 Quick Start

Start muLLM on port 6856, configure your providers, and open the web UI. Use a virtual environment with Python 3.11 or newer.

Path 1 — Install the package

python -m venv .venv
# Linux / macOS:
source .venv/bin/activate
# Windows PowerShell instead: .venv\Scripts\Activate.ps1
python -m pip install --upgrade mullm
mullm-server

Open the setup URL printed by the server. It uses HTTPS when a local certificate is available. The first interactive start opens setup automatically; installation itself starts no process. Set MULLM_OPEN_BROWSER=0 for headless operation.

Windows with local inference: connect your existing Ollama, TabbyAPI, vLLM, or llama.cpp runner in /setup. Mac with API keys only: add your provider keys and leave local inference disabled; MLX and model downloads are optional.

Use /chat for queries and /factory to follow projects, tasks, and human requests. The optional /classifier page previews local Kev.js inference. Factory guide · Upgrade and configuration guide.

The core install needs no separate constraints download. SQLite is the default cache. ChromaDB is an opt-in extra with unresolved advisories; see Security.

Optional installs, modes, and key storage

The base package includes the router, web UI, CLI, SQLite semantic cache, and cloud-provider clients. Add only the capabilities you need:

The setup page separates installs from feature toggles. Single User keeps private workstation defaults; Team / Proxy configures a stricter shared-gateway posture. Studio, 3D, Game Studio, ComfyUI, benchmarks, experimental backends, and research/developer features can be enabled independently. Some changes require a restart.

Provider keys can be saved to the OS keychain (recommended), Unix pass, or a local .env file with owner-only permissions. Best available selects the OS keychain when usable and otherwise falls back to .env. For containers and Kubernetes, inject keys through environment variables or mounted secrets rather than baking them into the image.

Path 2 — Rootless Podman

git clone https://github.com/mybrainrunslinux/mullm.git && cd mullm
podman build -t localhost/mullm:local .
podman volume create mullm-data
podman run --name mullm --replace -d \
  -p 127.0.0.1:6856:6856 \
  -v mullm-data:/data:Z \
  -e MULLM_REMOTE_ACCESS=true \
  localhost/mullm:local

This is a single muLLM container; run Ollama or another compatible inference backend separately. See Deployment for secrets, persistent state, and remote access.

Path 3 — Source / development

git clone https://github.com/mybrainrunslinux/mullm && cd mullm
python -m pip install -e ".[dev]"
cp config/mullm.toml.example mullm.toml   # configure providers as needed
mullm-server

For local generation, configure a supported inference runner in /setup. Ollama is one option (ollama pull qwen3-coder:30b); TabbyAPI, vLLM, llama.cpp, and MLX are also supported. Deterministic lookups need no model.

🚢 Production Deployment

# 1. Configure authentication and headless startup
export MULLM_DEV_MODE=false
export MULLM_API_KEY="your-long-random-server-token"
export MULLM_OPEN_BROWSER=0
# Add cloud provider keys through your normal secret store

# 2. Start muLLM
mullm-server

# 3. Verify
mullm --status

For concrete systemd, Podman, Kubernetes, and multi-replica guidance, see Deployment.

🖥️ CLI Reference

mullm "your question"                          # auto-route
mullm-server                                   # start server on :6856
mullm --status                                 # health check
mullm --cost                                   # session cost summary
mullm --stream "tell me a story"               # SSE streaming
mullm --model claude-haiku-5-5 "summarize X"   # force a specific model
mullm --tier cloud_full "complex question"     # force a specific tier

🌐 Supported Providers

The tier router directly supports Anthropic, OpenAI, Google, DeepSeek, GLM, and Cerebras. Additional providers have backend/gateway integrations; these are not all interchangeable with the tier router. Keys are optional. Model documentation reviewed 2026-11-05; account availability varies.

The router escalates through providers in cost order. Budget hard-stop at a configurable threshold prevents runaway spend.

💻 Hardware Compatibility

mμ|LLM scales from a laptop to a workstation. The local tier performance scales with your hardware; Tiers 0, 1, and 3 are hardware-independent.

Models are pulled automatically on first use when Ollama is the backend. For ExLlamaV2, MLX, and llama.cpp, point MULLM_OLLAMA_BASE_URL at their compatible API endpoint.

🧩 Module System

Each module is independently toggleable. Disable what you don't need to reduce dependencies and startup time.

⚙️ Configuration

All environment variables are prefixed MULLM_. Copy .env.example to .env and set only what you need.

🔌 API Reference

Base URL: http://localhost:6856

Core endpoints

# Route a query through all tiers
POST /query
{"content": "what is 2+2"}

# Split multi-part query, parallel execution
POST /query/split
{"content": "summarize X and then write tests for Y"}

# SSE streaming from local Ollama
GET /query/stream?content=tell+me+a+story

# Health check (Ollama + semantic cache availability)
GET /health

# Dashboard stats (cost, tier distribution, query counts)
GET /api/dashboard

# Live system stats (CPU, RAM, GPU, VRAM, loaded model)
GET /api/system

OpenAI-compatible endpoints

POST /v1/chat/completions   # drop-in replacement
GET  /v1/models             # model list

curl -X POST http://localhost:6856/query \
  -H "Content-Type: application/json" \
  -d '{"content": "what is 2+2"}'

{
  "response": "4",
  "tier": "groundtruth",
  "cost": 0.0,
  "latency_ms": 0.3,
  "model_used": "groundtruth-lut",
  "cached": false
}

🎯 Intent Categories

The classifier assigns every query to one of eight categories, each with a default routing strategy.

🔬 Research

mμ|LLM is informed by ongoing research. The manuscript and evaluation methodology are still under critical review; no preprint is currently published.

Key claims (verified on production data)

• 96.4% cost reduction — $27.87 vs $772 baseline, N = 15,363 queries, 22 days
• 87.5% free-tier resolution — Tiers 0–2 combined, zero cloud spend
• 100% HumanEval pass@1 at $0.00 — local tier only, no cloud escalation
• RouterBench AIQ 0.6673 — highest published score on this benchmark at time of submission
• DeBERTa-v3-small (44 MB) classifier achieves production-grade intent classification
• Four-tier waterfall architecture generalizes to any local + cloud backend combination

Citation details will be added when the manuscript is ready for publication.

📊 Benchmark Results

All results from a 22-day production run on real developer queries (N = 15,363). No cherry-picking, no synthetic benchmarks for the cost numbers.

🤝 Contributing

Contributions are welcome. Please open an issue before starting large changes.

git clone https://github.com/mybrainrunslinux/mullm.git
cd mullm
pip install -e .[all]
npx playwright test              # 59 free tests (no API calls)
npx playwright test --grep @spend  # paid cloud tests (explicit opt-in)

Guidelines:

• Tests must never spend money by default — cloud tests go behind --grep @spend
• Use Podman over Docker for container-based tests
• Cost accuracy matters — real API keys, real money
• Local-first — default to the cheapest capable tier

📄 License & Credits

License: Apache 2.0

Built by 0101 Technology

mμ|LLM is a research project and production system developed at 0101 Technology. The tardigrade mascot was chosen for its resilience — surviving extremes, running anywhere, needing nothing external.

Runtime privacy, deadlines, and model verification · Software Factory mode. Current models and custom classifiers · Upgrade guide.

Software Factory: mullm --factory factory.toml — configuration and acceptance gates.