Rigma

Local LLM deployment that tunes itself to your machine — a chat UI, an OpenAI-compatible endpoint, and an agent that can use tools.

Rigma probes your GPU and RAM, works out the model, quant and flags your hardware can actually hold, downloads a pinned llama.cpp build, and serves it.

Unlike generic runners, Rigma applies the tuning that actually matters per machine: MoE expert offload (--n-cpu-moe) sized to your RAM, architecture-aware KV-cache policies (e.g. q8_0 K-cache floor on DeltaNet-family models), backend selection per GPU generation (e.g. Vulkan over ROCm on RDNA4), flash attention, and session persistence (--slot-save-path) on by default. Every decision is auditable: rigma plan --explain shows the arithmetic and sources.

Quickstart (pre-alpha)

pip install rigma
rigma up                       # opens the chat UI; add a model from the Models page
rigma up --model <slug>        # probe, resolve, download that model, and serve it

A browser tab opens either way, and any OpenAI-compatible tool can use http://127.0.0.1:11500/v1 once a model is loaded. With no --model, up starts the UI without probing or downloading anything — a fresh install lands on an empty Models page, where picking a model runs the same probe-resolve-download path.

Chats persist server-side across restarts — the browser UI lists past sessions in a rail, renders markdown with fenced code, and supports regenerate. Each session carries its own system prompt (the registry ships sensible defaults per use case — general, creative, coding — so creative-writing models stay in character from the first message) and a per-session "use my documents" RAG toggle; retrieved sources appear inside the search_my_documents step in the transcript. A copy button and edit-last live in the legacy UI at /rizz.

Commands

rigma up flags: --model SLUG · --use-case coding · --port 11500 · --no-browser · --ctx N · --reasoning on|off|auto · --reasoning-budget N · --fa on|off|auto · --spec none|draft-mtp|… · --detach · --no-calibrate · --yes · --dry-run

Status

Pre-alpha (M5). rigma bench records machine-local calibration that outranks registry combos on your machine, and a failed launch automatically falls back (smaller quant → CPU floor) with each step explained. Combos come in two grades: verified (benchmarked on real hardware, evidence attached) and provisional (research-seeded fit math — run one and PR your numbers to rigma-registry). Verified so far:

The 9070 XT combos are keyed to the ram-16 tier. A machine with more RAM and no combo of its own falls back to the nearest lower tier, and rigma plan --explain names the tier it used — the reference box measures 31.4 GB (tier 32) and takes the ram-16 combo above.

Methods (set a chat up for the work, then automate it)

A Method configures a whole activity in one click — system prompt, sampler profile, thinking effort, tool posture and a Notes template — and then goes further: it carries rules, macros and workflows you can author yourself.

Six ship built in: Coding, Writing a book, Roleplay, Research, Learning, Organizing files.

• Macro — a button above the message box. Finish chapter summarises the chapter into your story bible and opens the next one.
• Rule — either standing guidance folded into the system prompt, or a trigger that fires on its own (after a chapter file changes, remind me to update the bible).
• Workflow — a named multi-step sequence.

All three are the same thing underneath: a step list. A step is one of tool, prompt, settings, note or new_chat, and steps can carry placeholders — {{var}}, {{last_reply}}, {{step:0}}, {{ask:Label}}, {{transcript}}, {{title_next}}, {{selection}}.

Make your own: hit + Create method in the Methods panel. That opens a chat where the model has only the method-building tools, so it can do nothing but help you build one — it asks a question, calls a tool, and the method takes shape beside the chat as it goes. Hit Save method when you like it.

Safety. A macro that only reads runs straight away. Anything that writes, moves, runs a command, or lets the model loose with its tools asks first, with a one-line preview naming the real files — Finish chapter: will run write_file on story_bible.md. Choose Always allow and it stops asking for that macro. That choice is stored on your machine and is never part of a shared method.

Sharing. A method is a JSON file in Rigma's home — ~/.rigma/methods/ (%USERPROFILE%\.rigma\methods\ on Windows). Export it, send it, import theirs. Importing never overwrites one of yours and never carries someone else's "always allow".

Skills (instructions you pull in mid-chat)

A Skill is a block of instructions you write once and drop into any chat by typing /name — no method to apply, no settings to change. Where a Method configures a whole activity, a Skill is a single reusable briefing: house style, a checklist, the shape of a file format you keep explaining.

/wildcard write me three variants

The skill's text goes in front of your ask for that turn only. Manage them in the Skills panel, or write them by hand: each one is a .md file in ~/.rigma/skills/ (%USERPROFILE%\.rigma\skills\ on Windows), so they live in your editor and your git repo like anything else. /skill:name works too, and a message that merely starts with a slash and names no skill — a path, a lone / — is sent exactly as you typed it.

In the chat

• Type ahead. Send a follow-up while a reply is still streaming and it queues; the model starts on it the moment the current turn ends. The queue is in memory, so it never survives a restart — a prompt waiting behind a generation stops meaning anything once the server is gone.
• Stop keeps what you got. Stopping a reply aborts the request and keeps the text already on screen, marked as a partial, instead of discarding it.

RAG (chat with your documents)

Rigma pairs with Raggity (AGPL-3.0, runs as a separate low-RAM process — ~300 MB) for grounded, cited answers from your own files through your tuned local model: pip install raggity[server], then rigma rag add <folder> and rigma rag ask "...". If raggity isn't on PATH, point RIGMA_RAGGITY_CMD at it.

Working on Rigma

git config core.hooksPath .githooks

One hook, and it matters: src/rigma/data/ui_v2/ is a built artifact that is committed, because the wheel ships it and an end user's machine never runs Node. Changing frontend-v2/src/ without npm run build therefore releases the old interface — and nothing else catches it, since pytest doesn't know the bundle exists and vitest tests the source. The hook refuses that commit and tells you what to run.

Running the tests

python -m pytest                  # everything except the hardware tests
python -m pytest -m hardware      # the opt-in live ones: a real GPU, real DSH turns

pyproject.toml puts src on sys.path (pythonpath) and excludes the hardware marker by default (addopts), so a fresh clone can run the suite without pip install -e ., and bare pytest means the same set CI runs. Two things are worth knowing:

• A subprocess does not inherit that. pythonpath is a pytest setting; a test that spawns sys.executable gets a fresh interpreter, which resolves rigma from site-packages if an older copy is installed there. CI installs editable (pip install -e .[dev]), so it cannot bite there; locally, pip install -e . is what makes those tests exercise the tree you are editing. The symptom when it bites: a subprocess test fails with ModuleNotFoundError for a module that plainly exists, or behaves like an older release.
• The hardware tests do real work — they drive DSH turns against a fake engine and need a GPU/model for others — which is why they are opt-in. pytest tests/test_harness_dsh_live.py -m hardware runs just the DSH ones.

License: Apache-2.0.