rigma 0.2.0

Hardware-aware local LLM deployment: probe, resolve, run.

Rigma

Hardware-aware local LLM deployment for consumer machines: rigma up probes your GPU/RAM, picks the community-verified best model + quant + flag combo for your exact hardware, downloads a pinned llama.cpp build and the model, and serves an OpenAI-compatible endpoint — no knob-mashing required.

Unlike generic runners, Rigma applies the tuning that actually matters per machine: MoE expert offload (--n-cpu-moe) sized to your RAM, architecture-aware KV-cache policies (e.g. q8_0 K-cache floor on DeltaNet-family models), backend selection per GPU generation (e.g. Vulkan over ROCm on RDNA4), flash attention, and session persistence (--slot-save-path) on by default. Every decision is auditable: rigma plan --explain shows the arithmetic and sources.

Quickstart (pre-alpha)

pip install rigma
rigma up            # probes your machine, downloads the best model, opens the chat UI

That's it — a browser tab opens with a chat connected to your tuned local model, and any OpenAI-compatible tool can use http://127.0.0.1:11500/v1.

Commands

Command	What it does
rigma up	Start everything; opens the chat UI in your browser
rigma chat	Chat with the running model in the terminal
rigma status	What's running, where
rigma stop	Stop the model server and UI
rigma models	What fits your machine
rigma plan --explain	What up would run, with the math
rigma doctor	What Rigma detects on this machine

rigma up flags: --use-case coding · --model SLUG · --port 11500 · --no-browser · --turbo (fast download, may hog your bandwidth) · --yes · --dry-run

Status

Pre-alpha (M1). Verified combos:

Hardware	Model	Backend	Result
RX 9070 XT 16GB + 16GB RAM (Windows)	Qwen3.6-35B-A3B UD-Q3_K_XL, ctx 32K, n_cpu_moe 10	Vulkan (llama.cpp b9867)	verified 2026-07-06: 57.1 t/s gen, 689 t/s prefill @ 4K prompt

Design: docs/superpowers/specs/2026-07-03-rigma-design.md. License: Apache-2.0. RAG integration (via Raggity, AGPL-3.0, separate process) lands in M4.