solvi

Build decision systems from a catalog of Python functions and checks plus typed questions, and get answers you can verify.

Why

You describe a task with plain Python functions (computations, checks, answer rules) and questions with typed answers (yes/no, a choice, a score, "not stated", a span of the text, a ranking, a number range). For each request, a strategist plans which functions and checks to run for the asked questions. Every answer comes with:

• a confidence (calibratable per question);
• a reason you can check: the rule inputs, a formula over computed facts, or a quote with character offsets in the source document;
• a hash-chained trace of every step, which can be re-executed later to confirm the answer or pinpoint the step that was altered.

When something cannot be computed, a function fails, or a rule returns an answer outside the allowed options, solvi abstains instead of guessing. A failed hard check always overrides any model confidence.

Grounded decisions. Fuzzy proposes, deterministic decides, everything is in the trace. Each fact and answer records its provenance — given, computed, quoted, decided (a model's choice among options, with probabilities) or learned — and model-backed steps record the model's id and fingerprint, so a replay can tell when the model changed since a decision. A model's quote that is not literally the text at its offsets, or a choice outside its options, is rejected and counted (system.stats); a fallback producer runs or the question abstains. print(res.audit()) shows what each answer rests on, which safeguards fired, and how much of its support is deterministic. A decision without models and one with models are the same system (examples/12_grounded_audit.py).

And it is fast. The strategist plans a flow over a 10 000-part catalog in about 11 ms and runs only the parts the questions need (2.3% of that catalog). Hard checks run first, so a failing one skips the expensive rest; independent slow parts (API calls, model inference) run in parallel. On an insurance-claim desk with slow services (examples/09_strategy_at_scale.py) a full decision takes 461 ms instead of 1 122 ms for a script that computes everything, and 152 ms when an expired policy settles the claim first.

Try it

• solvi playground: write a decision task in Python and run it, watch the strategist's plan, tamper with a trace and see the replay catch it, learn rules from examples.
• solvi documents: cited, typed answers from contracts, invoices, receipts, leases and more — the ModernBERT extractor (ONNX) and the decisions both run in your browser; add a field by describing it.
• solvi arcade: game agents that explain every move — tic-tac-toe, maze, minesweeper, 20 questions, Mafia detective, a bot arena, and "hack the trace".
• solvi realms: an endless strategy game whose factions are solvi systems — tested for 100 000 turns: flat decision time (~0.3–0.6 ms), bounded memory and state, every sampled trace replay OK.
• All run entirely in your browser (Pyodide): no server, no GPU, nothing you type leaves the page.
• Models: solvi-ai/solvi-large (typed decisions, 396M, preview), solvi-ai/solvi-base (the same answers on a CPU / in ONNX, 150M, preview), solvi-ai/extract-base (fields by description) and solvi-ai/extract-receipts. Each model card states what the model was measured on, how well it does, and its limits; all models are listed at huggingface.co/solvi-ai.

Gallery

gallery/ — fifteen decision tasks across directions (support triage, email routing, content guard, security alerts, AI-agent audit, release rollout, KYC/AML, clinical screening, credit with adverse-action reasons, procurement 3-way match, double-charge refunds, predictive maintenance), each with scenarios, a runner and a side-by-side against an answer-only model. Three are helpers for coding agents: a pre-edit rule check (allow / block / escalate a file write), review triage (quick review only when seven risk questions are a confident "no", with a stated bound on risky changes that slip through) and a skill picker with an honest "none".

Install

pip install solvi              # core: rules, checks, learned answer heads (numpy, scipy, pydantic)
pip install "solvi[model]"     # + torch, transformers: ModernBERT field extractors for documents and the decider
pip install "solvi[onnx]"      # + onnxruntime, tokenizers: the decider (solvi.decide) on CPU without torch
pip install "solvi[serve]"     # + fastapi, uvicorn: `solvi serve app.py:system` — the questions over HTTP (also --mcp)
pip install "solvi[mcp]"       # + the official MCP SDK for solvi serve --mcp (without it, a built-in stdio server)
pip install "solvi[otel]"      # + opentelemetry: decisions as OpenTelemetry spans (solvi.otel)
pip install "solvi[duckdb]"    # + duckdb: stored decisions in a DuckDB file (solvi.DuckDBStorage); [postgres] for PostgreSQL
pip install "solvi[lora]"      # + torch, transformers, peft: part.adapt_lora, a LoRA adapter per question (experimental)

Two words mark what is not settled yet: preview — it works and is tested, and its API may still change; experimental — no published model or measurement backs it yet, and it may change or go.

solvi.agents (guarding an agent's tool calls) needs only the core; its adapters use the PydanticAI, LangGraph or OpenAI Agents SDK you already have ("solvi[pydantic-ai]", "solvi[langgraph]", "solvi[openai-agents]" install them).

Requires Python 3.10+.

Quickstart (core only, no model)

from datetime import date
from solvi import Answer, Catalog, Question, System

cat = Catalog()

@cat.fn                                            # argument names = facts it reads; function name = fact it sets
def days_requested(start, end):
    return (end - start).days + 1

@cat.fn
def remaining_after(balance, days_requested):
    return balance - days_requested

@cat.check(hard=True, then={"approve": "reject"})  # if this check is False, "approve" is forced to "reject"
def enough_balance(remaining_after):
    return remaining_after >= 0

@cat.check
def enough_notice(start, today, days_requested):
    return days_requested < 5 or (start - today).days >= 14

@cat.rule("approve")
def approve(enough_notice):
    return "approve" if enough_notice else "needs_manager"

system = System(cat, [Question("approve", "Approve the leave?",
                               Answer.choice(["approve", "needs_manager", "reject"]),
                               requires=["enough_balance"])])
res = system.ask({"start": date(2026, 10, 19), "end": date(2026, 10, 23),
                  "today": date(2026, 9, 25), "balance": 14})
print(res["approve"].answer, res["approve"].confidence, res["approve"].why)
print(res.state_text())
print(res.trace.replay(system))

Output:

approve 1.0 enough_notice = True
days_requested           = 5
remaining_after          = 9
enough_balance           = True
enough_notice            = True
{'ok': True, 'steps': 5, 'mismatches': [], 'models': [], 'answers': 'same', 'catalog': 'same'}

With "balance": 3 the hard check fails and the answer is reject with status == "forced", whatever the rule says. solvi.show.show(res, cat) prints answers, the planned flow, the computed state and the replay result in one go.

Command line

solvi init triage --with-model && cd triage     # a typed catalog, passing cases.json, README, CI workflow
solvi test . && solvi check catalog.py:system   # regression cases and the catalog lint (what CI runs)
solvi ask catalog.py:system example.json --audit            # one decision and what it rests on (--json, --report html)
solvi models pull solvi-ai/solvi-base           # a local decider for offline use; `solvi models` lists, `check` measures
solvi calibrate catalog.py:system route labels.csv --risk 0.1   # act_guard → route.calib.json, loaded by the catalog
solvi hook install                              # Claude Code's edits checked against .claude/solvi-rules.toml

Every command is in the guide.

How it works

• Catalog. @cat.fn (computation), @cat.check (bool), @cat.extract (value from text, returned as a Quote with offsets) and @cat.rule(question) (answer rule). A part's contract is its signature: argument names are the facts it reads, the function name is the fact it sets. Type hints are optional and become the facts' types (def risk_score(risk_points: dict[str, float]) -> float): producer and consumer types are checked when a part is registered, values are validated / coerced with pydantic at run time, and a value that fails is rejected like an ungrounded quote (safeguard type_rejected). Untyped parts cost nothing.
• Questions. Question(name, text, Answer.yes_no() | Answer.choice([...]), requires=[...]). Questions without a rule get a small answer head trained from labeled examples (system.fit) or a readable learned rule list (system.learn_rule).
• Strategist. For each question it walks backwards from the rule's arguments (or the learned features) through the catalog signatures to the keys of init_state, adds the question's required parts (requires) and every check that touches a computed fact. Everything else in the catalog is not executed; the flow records why each part was taken or skipped.
• Execution. Each part runs once, even if several questions need it. Hard checks and their inputs run first; when one fails, the steps only the settled questions needed are skipped (res.trace.skipped). With System(..., workers=8) independent steps run in parallel threads as soon as their inputs are ready. Results go into the computed state (res.values, res.state_text()) with their provenance; extracted values keep their quote. Each step record is hashed and chained to the previous one in flow order, so the trace does not depend on scheduling.
• Answers and trace. res[q].answer / .confidence / .why / .status (ok, forced, abstain), plus res.trace.replay(system), which recomputes every step from recorded inputs and reports mismatches, broken hash links and quotes outside the text — and, given the System, a stored answer that is not the one the trace gives.

Typed decisions with any model

Types declare questions; a model proposes; checks decide. The fields of a pydantic model are the questions, their types the kinds (one option, several, an ordered score, yes/no); a decider answers them about a text or a JSON / pydantic state with probabilities, a calibrated confidence and act / escalate, and hard checks, constraints and rules still decide. The decider is whichever model you have:

• An LLM — solvi.llm.llm(base_url, model, api_key=...): any OpenAI-compatible chat-completions server (OpenAI, OpenRouter, vLLM, llama.cpp, Ollama). Nothing to install beyond the core; its JSON replies are validated, and an invalid one escalates, never a guess.
• A decision service — solvi.systemone.systemone(url, model): anything that speaks the System One API.
• A local checkpoint, for offline or cheap cases — DecideModel.load("solvi-ai/solvi-base") (solvi[onnx], no GPU). solvi-base is a 150M ModernBERT-base cross-encoder distilled from solvi-large: about 50 ms per question on a CPU (ONNX fp16, 4 threads). Its card is honest about where it stands: 54.5% zero-shot on typed questions over JSON states (the same as solvi-large), 56.3% on Fast Decisions dev — not better than earlier small models there — and 0.602 on the jabr classifier benchmark, where Jev reaches 0.966. A preview: fit it on 30–60 labelled examples of your task and calibrate its escalation on your own stream before relying on it.

import os
from typing import Literal
from pydantic import BaseModel, Field
from solvi import Catalog, Scale, System
from solvi.llm import llm

class Triage(BaseModel):
    team: Literal["billing", "technical", "shipping"] = Field(description="Which team should handle this ticket?")
    urgency: Scale[Literal["low", "medium", "high", "critical"]] = Field(description="How urgent is it?")
    angry: bool = Field(description="Is the customer angry?")
    topics: list[Literal["refund", "delay", "bug"]] = Field(description="What does the ticket mention?")

model = llm("https://api.openai.com/v1", "gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"])
# offline: model = solvi.decide.DecideModel.load("solvi-ai/solvi-base")   — the same questions, the same checks
cat = Catalog()
questions = model.questions(cat, Triage, text_fact="ticket", min_confidence=0.6)

@cat.check(hard=True, then={"urgency": "critical"})         # a legal threat is critical, whatever the model says
def no_legal_threat(ticket) -> bool:
    return "lawyer" not in str(ticket).lower()
questions[1].requires.append("no_legal_threat")

res = System(cat, questions).ask({"ticket": {"subject": "Charged twice", "body": "Refund my double payment!",
                                             "customer": {"tier": "pro"}}})
print({q: (r.answer, r.status) for q, r in res.results.items()})
print(res.audit("team"))           # the probabilities, the model's fingerprint, what escalated and why

A state is read as key paths (customer.tier: pro); an unsure or escalated answer abstains with the reason (system.stats["model_escalated"], ["low_confidence"]). examples/15_typed_decisions.py runs the whole story with a stand-in model, without network. Whatever the model, read what it was measured on before relying on it, and calibrate it on your own labelled stream (part.act_guard, below): checks, constraints and escalation are what make the answers safe to act on, not the model alone. The checkpoint contract of a local decider is in docs/decide_format.md; a local checkpoint answers one question per forward pass with the published checkpoints, several with DecideModel.load(..., multi_question=True).

Every answer is a value and a confidence, and the types also declare answer primitives: Maybe[T] ("not stated" — solvi.Unknown — is a real answer, unlike an abstention), Span[float] (an exact piece of the text, parsed), Rank[...] (the top k, in order), Estimate[0, 7, 14] (a number with an interval), and evidence quotes on any answer (Claim(value, evidence=[...]), Question(require_evidence=True)) — each checked in the text, from rules or a model (guide, examples/16_primitives.py).

Escalation with a guarantee, several models, serving

• A guaranteed risk. part.act_guard(examples, max_risk=0.10) calibrates on a few hundred labelled examples of your stream so that P(answered alone and wrong) ≤ 10% for inputs like them (conformal risk control) — a share of all inputs, not the error among the answers given alone (calibrate_for(max_error=0.10, method="ltt") bounds that); the audit shows the promise behind every answer, or says there is none. part.conformal(examples) gives the person who takes an escalation a short list of candidates. Near ties escalate (min_margin=), and the answer does not depend on the order the options are listed in (sorted by default).
• Several models. Any decider above — an LLM, a System One service (Jev, Kev, Von, Laya-serve, …), a local checkpoint — combines with the others: Cascade, Vote and Route (solvi.multi) combine models — the next model only when one escalates, an answer only when models of different families agree, or a model picked by code — under one guarantee (examples/20_vote_across_families.py shows a vote with stand-in servers).
• Serving and operations. solvi serve module:system exposes the questions over HTTP (OpenAPI from the same types), MCP and the System One API; await system.aask(...) runs async parts concurrently with timeouts; cost_policy="measured" lets the planner pick the fastest equivalent source and switch when it slows down. TraceStorage keeps decisions with a hash chain across them; solvi diff shows which stored decisions a rule or model change would flip; solvi test, solvi check and the honesty suite (solvi honesty) belong in CI; res.report(format="html") and solvi report decisions.db --html out.html give an auditor one page per decision or per period, and solvi.otel.export(res) puts every step in your OpenTelemetry traces. res.counterfactual("approve") says what would have changed the answer ("approve if amount ≤ 1000 (now 1200)"), re-running only the code with the models' recorded proposals held.
• Guarding an agent's tool calls (preview). The agent proposes {"name": tool, "arguments": {...}}; solvi.agents.Guard checks it — the tool is in the catalog, the arguments validate against its types, the values that must come from the conversation are quoted there (and not only from a tool output that says "ignore previous instructions"), your policies (limits, roles, allow-lists) are ordinary hard checks, and an optional decider asks "did the user ask for this?" under act_guard and perturb — then allows it (solvi runs the function), denies it with the reasons, or escalates it to a person. Every decision is a stored, replayable trace. Adapters for PydanticAI, LangGraph and the OpenAI Agents SDK, and solvi serve --guard catalog.py:guard --upstream CMD in front of an MCP server (guide, examples/19_agent_guard.py).
• Behind a coding agent's hooks (preview). solvi hook install puts solvi in front of Claude Code's edits and prompts: every Edit / Write is checked against a rules file (forbidden patterns, required functions, Python calls read from the code; fuzzy questions for a model, which block only with a calibration) and denied with the rule and the lines, sent to the user, or let through; a prompt gets the one project skill it needs, or nothing. Deterministic by default; every decision stored and verifiable; Codex as a preview (guide, examples/22_coding_agent_hooks.py).
• Text in. system.ask_text("please refund order A-10457, 1.5 million RUB, paid on 12 September 2026", textin=TextIn(system, decider, patterns={"order_id": r"A-\d+"})): the decider picks which question the message asks (or escalates when unsure), each input field is read with a quote (found by a deterministic cue finder — chosen over the checkpoint's span pointer by measurement) and a deterministic parser (numbers, dates, enums, yes / no; a date without a year is not guessed), missing required fields are listed for a clarifying question, and the trace says those values were read by a model, not given. solvi serve answers texts at POST /ask_text and as the MCP tool ask_text.
• Long documents. decider.decision(..., long="retrieve"): a contract longer than the decider reads is split into sections, BM25 picks the few that bear on the question, the decider reads only those, and quotes point into the whole document; the trace lists the sections read. long="full" reads a text whole up to the length a checkpoint trained on long inputs declares (max_len_long), and retrieves within that length beyond it.
• Learning from corrections. part.memory() escalates an answer when similar corrected cases say another one; fit heads refit on all kept examples as corrections accumulate; part.adapt_lora(examples, holdout=0.3) trains a small LoRA adapter for one question on solvi-base once it has ~100 labelled answers (solvi[lora], experimental); System.learning(store) proposes updates from trusted corrections only and promotes one when it passes held-out, honesty and calibration gates, with rollback (experimental, off unless called) (guide).
• Records you can check later. store.signature() — 64 bytes kept next to the chain's head — later names the one stored record that was edited and restores its hash (preview). solvi.charts draws a chart in which every number is quoted from the text and checked (unit, scale, a pie that adds up), as a deterministic SVG that replays to the same bytes (preview; examples/21_verified_chart.py). The audit, show and the safeguard report render in Russian with System(..., lang="ru").

Planning around dead ends and costs (code strategist)

The default strategist needs the inputs of every producer of a fact. solvi.strategy.CostStrategist() plans around producers whose inputs are never given, and with producers="equivalent" picks the cheapest verified plan by declared cost= (an exact 0/1 program; hard checks that govern a question always stay in the plan). No model is involved; the plan is one hashed record in the trace and replay re-verifies it.

from solvi.strategy import CostStrategist
system = System(cat, questions, strategist=CostStrategist(producers="equivalent"))

A segment model that proposes producers when costs are not declared, and solvi.aliases (wiring parameter names that match no fact), ship as experimental; their weights are not published. See docs/strategist.md and examples/17_model_strategist.py.

Extract from documents

With solvi[model], fields are found by a fine-tuned ModernBERT extractor. The extracted value is always a span of the document, so every answer built on it can be cited.

from solvi.extract_long import LongSpanExtractor

ex = LongSpanExtractor.load("solvi-ai/extract-base")      # a field is a description; long texts in 1024-token windows
ex.fit([(text, "the total amount paid", (start, end)), ...], epochs=3)   # a few dozen labeled documents of your task
ex.save("my-extractor")
cat.extract(ex.field("total", "the total amount paid"))   # doc -> Quote(text, start, end, confidence), or no answer
cat.extract(ex.field("date", "the date of the purchase"))

@cat.fn
def amount(total):                                        # extracted values are strings; parse them in ordinary functions
    return float(total.replace(",", ""))

LongSpanExtractor reads the whole text in overlapping windows and supports "no answer" with a per-field threshold (ex.tune_threshold(name, held_out)). What the published solvi-ai/extract-base does without labels of your task, from its model card (held-out fields and data sets, one seed): CORD receipt fields it was never trained on, 46.5% and 67.9%; five never-trained CUAD clause types, 73.1%; Kleister-NDA and SROIE, never seen, 2–95% by field (addresses 2%). With per-field thresholds from 40 labeled contracts it reached 85.5% on CUAD; fine-tuned on 25–100 SROIE receipts, 86–89%. So describing a field is a start, not a finished extractor: label 25–100 documents and fine-tune. solvi.extract_multi.MultiSpanExtractor (a fixed field list, one pass per document for all fields) is the extractor behind the receipt numbers below; it has no save / load. See docs/guide.md.

Results

Every number in this README comes from a script in benchmarks/ or from a published model card. Field extraction: the extract-receipts card reports 97.3% on typed questions over CORD receipts (100 test receipts) with ECE 0.011 and 98.6% of questions answered at ≥ 99% precision, and the extract-base card its zero-shot and few-label numbers; the decision models' cards (solvi-base, solvi-large) theirs. On the gallery, the rules, quotes and checks are the answer: every answer is backed by a quote at stated offsets, a computed fact, or the system abstains. Details: docs/benchmarks.md.

Nine public tasks

Real tasks on public data, each with a baseline that does not use solvi and one solution with solvi, chosen on dev and scored on a held-out split (benchmarks/tasks/, solvi 0.8.0, gpt-oss-120b where an LLM is used):

solvi does not make a model more accurate: on RAGTruth, BIRD, τ-bench and NAB it did not beat the baseline. What it added is the promise on what is answered alone, consistency across items, a search where a model guessed, and a stored, replayable record of every decision. German Credit (rules and an audit of a rule change) decides exactly as plain code does and adds the audit. Every number, its caveats and the cost of a run: benchmarks/tasks/README.md.

How it compares to asking an LLM

We gave the same inputs and written rules to solvi and to four LLMs: Grok 4.7, gpt-oss-120b, Qwen3-235B-2507 and DeepSeek-V3.2. The sets were the gallery, generated refund and 3-way-match cases built to trip models up, and Banking77 message routing. Strong reasoning LLMs followed short written rules as accurately as solvi. solvi was not more accurate there. What differs is everything around accuracy:

Inside solvi, no LLM broke a hard check, and wrong answers given without a person stayed within the promised 10%. On free text the LLMs win: 0.925-0.972 on bank messages against 0.675 for solvi-large. A hosted decision model, Jev by TypeSafe, read bank messages almost as well as the best LLM (0.964) but scored 0.819 on refunds with 53 limit violations; inside solvi it broke no hard check either. An open one that reasons first, Jeeves by PostHog, came closer on the rules (0.863 on refunds, 0.912 on 3-way match) but still broke hard checks when asked directly. Full tables, caveats and a reproducible runner with every raw answer: docs/vs_llm.md, benchmarks/vs_llm/.

Speed

One ask without a model (benchmarks/ask_speed.py: planning, running the parts and hashing the trace; Python 3.10, Intel i7-12700H shared with other jobs, medians of three runs of 15 passes):

A large input costs its hashing: every given value is written as canonical JSON and hashed once per ask (about 0.25 µs per float). The slowest gallery entry checks its texts for instruction-like sentences (perturb), which takes most of its time. Saving each response to a store adds its own cost (the whole response is written as JSON).

Strategist on random layered catalogs (benchmarks/strategist_scale.py, one CPU core; solvi 0.7.1 with the changes since, Python 3.10, Intel i7-12700H, medians of two runs). The parts are trivial arithmetic: running all of them ("run all") costs less than one ask, so the saving shows only when parts are slow (services, models), as in the insurance desk below.

Insurance claim desk with six slow services of 100-300 ms (examples/09_strategy_at_scale.py):

A rule-only decision on a small catalog takes well under a millisecond (benchmarks/ask_speed.py); on documents the extractor dominates.

When to use it

• The answer is a computation or a rule over a few values found in a document or a record: receipts, invoices, contracts, requests, orders.
• You need to show why: auditors, compliance, or a human reviewing low-confidence cases.
• Some rules are non-negotiable (hard checks), and the rest can be learned from about 100 labeled examples.

When not to use it

• Open-ended free-text questions or generated answers. solvi answers typed questions only: yes/no, choices, scores, multi-label, "not stated", exact spans of the text, rankings and number ranges. Around a model that writes (a query, a plan, a JSON extraction) it checks, compares and re-asks — solvi.generate, solvi.agree, solvi.refine — but does not make the writing better. It searches for a plan only over a space you enumerate (solvi.search).
• New fields with no labeled examples. Extracting a field from its description alone is not reliable yet: the published extract-base gets 2–95% by field on data sets it never saw (see "Extract from documents"); label 25–100 documents and fine-tune.
• No labels at all. Plan on roughly 100 labeled documents (field positions) per task.
• CPU-only deployment with a quantized extractor: dynamic int8 quantization changed half of extract-base's spans (its model card), so none is provided; use fp32 or fp16 and measure the time per document on your hardware.

Examples

Run them from a clone: python examples/01_leave_request.py.

More

• docs/guide.md: full API walkthrough.
• docs/best_practices.md: what we learned while building on solvi, as advice.
• docs/decide_format.md: the decider checkpoint contract (for training your own).
• docs/strategist.md: the code strategist and the experimental model strategist and name matching.
• ROADMAP.md, CHANGELOG.md, CONTRIBUTING.md, SECURITY.md.
• docs/benchmarks.md: the scripts behind the numbers, what solvi adds to an ask, the dataset loaders.
• docs/vs_llm.md: solvi vs asking an LLM (Grok 4.7, gpt-oss-120b, Qwen3, DeepSeek), with raw answers.
• benchmarks/: the benchmark scripts that can be rerun, and the dataset loaders behind the extraction numbers (SROIE, CORD, CUAD, Kleister-NDA — their scripts are not in the repository).
• Tests: pytest.

License

Apache-2.0. See LICENSE.

Citation

A paper is in preparation. Until then, please cite the repository:

@software{solvi,
  title  = {solvi: verifiable decision systems from catalogs of functions and checks},
  author = {mxkuzn and solvi contributors},
  year   = {2026},
  url    = {https://github.com/solvi-ai/solvi}
}