Decisio

Jev-level decisions on your own hardware, open source.

Decisio is an open-source server that answers typed questions about any text, with a calibrated probability for every option, from one forward pass and no generated text.

Second among open-weights systems on JevBench, level with Jev, and the highest-ranked with frozen weights. (JevBench v1.6.1, decisio on Gemma 4 31B; how we read the board.)

How it works (illustration; the probabilities are examples, real recorded responses are below).

Why Decisio

• Higher than Jev on what the model knows. Intelligence axis: 70.4 against Jev 1.13.0's 63.6.
• Second on the sealed set. Sealed-set intelligence: 73.3 against Jev 61.6, H2O-Lightning-4B 59.0 and Quyet-1.0-Large 71.6.
• Several questions on one read. A second question on a 3,000-token state already read takes 43.3 ms (one RTX PRO 6000 at 585 W).
• Long prompts, many options. Prompts up to 32,768 tokens and 255 options per question.
• Your hardware, no per-token bill. It runs on your own GPU, Mac or laptop; the cost is the card, not a bill per token.

One RTX PRO 6000 at 585 W; intervals and the rest of the figures are under Benchmarks.

Benchmarks

Each base with its own defaults on one RTX PRO 6000 Blackwell: the Qwen and Gemma 4 12B bases in one session (runs/2026-10-04_gemma-base/), Gemma 4 31B in its own (runs/2026-10-04_gemma-4-31b/), image input in another (runs/2026-09-27_image-input/). Measured by us with the public harnesses (the Decision Index kit 0.2.1) and recorded in runs/; none is a board score.

Notes: Decision Index figures are 0.2.1; the JevBench row is the 231 published items; intent heads are six draws (Qwen, 12B) and three (31B); image input is ImajevBench v2.0-lite, the 230 answerable items; n/m is not measured; the latency rows are server time on 0.8.1's served defaults with the engine in the server's process.

The latency rows were measured in one session on decisio 0.8.1's served defaults: the engine in the server's process, and on the Gemma bases a single question on a new state registering its boundary first (runs/2026-10-06_latency-585w/). The card was one RTX PRO 6000 Blackwell Workstation Edition with its power limit at 585 W (default 600 W), on an AMD Ryzen Threadripper 9960X host. On the Gemma bases that limit held the clock back during most first reads, so a card at 600 W may read new states faster; on the Qwen base it did not. In the same session, the boundary registration added +16 to +28 ms (12B) and +21 to +34 ms (31B) to the first read of a new state of 300 to 3,000 tokens, and a second, different question then read the state from the cache on 20 of 20 states per base. Changes paired within one session:

• 0.9.0 to 0.11.0, Gemma 4 12B registered a state's boundary after the answer (before again since 0.11.1, because requests sent back to back waited behind the previous registration): in the same session its first reads with the queue idle took 33.3, 90.3 and 237.1 ms at 300, 1,000 and 3,000 tokens, against 56.7, 107.7 and 261.9 with the registration first; Gemma 4 31B keeps registering first, since after the answer cost it a quarter to two fifths of its throughput under load (runs/2026-10-06_latency-585w/register_boundary.md, docs/running.md).
• 0.8.0, the engine in the server's process: on an AMD EPYC 7452 host (power limit not recorded), the Qwen base answered faster in every cell, by 3.8 ms on a cached question and by 0.6 to 32.2 ms on new states (runs/2026-10-05_engine-death-gates/). On that host the Qwen base took 1.4 to 2.1 times the records of 2026-10-02 (engine in its own process, host CPU and power limit not recorded) in either arrangement, while the Gemma bases did not.

Absolute latency depends on the host's CPU and on the card's power limit (docs/running.md).

docs/comparison.md sets all three bases beside Jev and the leading open entries on every Decision Index benchmark, JevBench's published questions, latency, cost and capabilities (the public board's figures and their date are in EVAL_CARD.md section 8.1). The intent heads use labelled examples, so their figures are not comparable with zero-shot systems. Calibration on JevBench, as ECE on the standard and hard tiers: 0.121 and 0.043 on the Qwen base, 0.033 and 0.085 on the Gemma 4 12B base, 0.035 and 0.091 on the Gemma 4 31B base. EVAL_CARD.md has the full tables, the calibration figures and the disclosures of what was fitted on what (sections 4, 6.4 and 7.4).

JevBench, axis by axis

• Among the ten highest open-weights systems and Jev: 2nd overall, 2nd on sealed questions (73.3, never published) and 3rd on intelligence (70.4, ahead of Jev's 63.6).
• Within 2 points of the highest on calibration (88.7; Jev 90.6) and on speed (91.2; H2O-Lightning-4B 92.6).
• 9th on the board's cost axis (51.7; H2O-Lightning-4B 60.3 is highest). On your own hardware you pay for the card, not per token.

Many questions about one document

Fifty questions about a 3,000-token document in 1.50 s on one RTX PRO 6000; the large open decision models that re-read the document for every question take 45.7 to 67.9 s. At 50 questions it is also faster than the two systems that share the document through the prefix cache: 1.50 s against 2.35 s for Blink v0.3 and 2.39 s for H2O-Lightning-4B.

How it was measured, by Tachara AI Lab on one NVIDIA RTX PRO 6000 Blackwell at 600 W, each system on its own published server:

• decisio v0.10.0 with vLLM 0.30.0 in its default mode (31B warm); Blink on stock vLLM 0.30.0.
• Synthetic support tickets of exactly 3,000 and 6,000 tokens (Gemma 4 tokenizer), the same documents for every system; one fixed pool of 50 questions (20 choice with 4 options, 20 yes/no, 10 score with 4 levels).
• Medians of the wall time of everything asked about one document, over 10 documents for decisio, Blink and H2O-Lightning-4B and 5 for the others; this compares time, not accuracy.
• Blink v0.3 is a 26B-A4B model (4B active parameters, 17.5 GB), not a 4B model; H2O-Lightning-4B is a 4B. Both take one question per request and share the document through vLLM's prefix cache. At 3,000 tokens only; at 6,000 tokens the 31B is also lower than both (2.39 s against 3.48 s for Blink and 3.07 s for H2O).
• Quyet-1.0-Large has no 6,000-token cells: they were not reached within its 15-minute cap.

How we read the board

Decisio scores higher on the sealed items than on the open ones.

• Source. The figures are the board's, from its published file for JevBench v1.6.1 (sha256 5cd8c1332226..., read 2026-10-09), row decisio-gemma-4-31b-v080; the charts are drawn from that file, view A.
• Open weights. Second among the board's 128 open-weights systems: the board's own open_board_rank is 2, and H2O-Lightning-4B ranks above us (72.52, interval 69.79 to 73.79) with an interval that overlaps ours (69.86 to 72.81). Overall the row is fourth of 135 on the headline composite (third on view B, fourth on view C).
• Frozen weights. The board has no frozen-weights class. We count a row as frozen when its display text says so (or says stock Gemma or Qwen) and names no LoRA, fine-tune, merge, training, adapter, head or decoder. The claim is about rank: decisio's row is the highest-ranked of those.
• Level with Jev. Our interval (69.86 to 72.81) and Jev 1.13.0's (69.14 to 72.40) overlap; the scores are 71.69 and 71.49.
• Intervals. Two other frozen-weights rows have intervals that overlap ours, decider-12b-v1 (rank 9) and Cygnet (rank 13); deck-31B (rank 12) does not.
• The row. It is decisio v0.8.0 with Google's weights quantized to FP8 on load, measured by the board on one H100 80 GB on 2026-10-06; 1,477 of its 1,500 items were answered, the 23 others being items of about 80,000 tokens, over the 32,768-token context. The 31B repository's FP8 weights are that same quantization, stored.
• Not fitted, but read. Our temperatures were not fitted on JevBench; we have read its published items (EVAL_CARD 7.4).
• Also on the board: second on the sealed set.

Quickstart

Ollama on a laptop, then pip on a GPU, then Docker.

On a laptop, with Ollama

The Gemma listing's smallest tag takes 8.6 GB in Ollama (measured on a 64 GB Mac), so it should fit a 16 GB laptop (docs/running.md):

ollama pull aminroudaki/decisio-gemma      # Gemma 4 12B, 8.6 GB in Ollama at its smallest tag
curl http://localhost:11434/v1/systemone -d '{
  "model": "aminroudaki/decisio-gemma",
  "state": "Hi, since this morning none of our 40 staff can log in to the dashboard. We get \"session expired\" right after entering the password. Payroll is due today.",
  "questions": {
    "urgent": {"type": "noul", "instructions": "Does this ticket need a response within the hour?"},
    "category": {"type": "choice", "instructions": "Which team should handle this ticket?",
                 "criteria": {"billing": "Invoices, payments, refunds", "access": "Login, passwords, permissions", "bug": "The product behaves wrongly", "other": null}}
  }
}'

Ollama builds its own prompt and applies no calibration, so its numbers are the ones on its model page, not the ones in this repository.

On a GPU, with pip

One NVIDIA card with 96 GB (the size everything below was measured on), Linux, Python 3.12:

python3.12 -m venv .venv && . .venv/bin/activate
pip install "decisio[serve]"
python -m decisio.serve.vllm_engine          # serves Gemma 4 31B; the first start downloads about 31 GB
curl -s http://127.0.0.1:8000/health         # answers once it is ready

The measured environment is the lockfile's (uv sync --extra serve --frozen, GPU server); pip resolves its own versions around vLLM 0.30.0. Measured by Lab 2 on 2026-10-09 on decisio 0.11.0 from PyPI, on one RTX PRO 6000 Blackwell at 600 W and an AMD EPYC 9534: pip install "decisio[serve]" took 124 s; the first start of the no-flag 31B was ready after 7 min 54 s, including the download of its weights, and a start with the weights already on disk after 4 min 3 s; the first start of --base gemma-4-12b after 4 min 12 s, including its download (these are one session's times, with the network that session had).

With Docker

docker run -d --gpus all --shm-size 8g -p 127.0.0.1:8000:8000 -v decisio-data:/data ghcr.io/aminry/decisio:0.11.0

The release notes give the image's digest; Docker has the compose file. The same curl as above, with /v1/systemone on port 8000 and no model field, asks the server you started.

Three decisions, as recorded

Each request is what was sent; each response is what the server answered on the Gemma 4 31B base (probabilities rounded to three decimals here). They are the three requests of this README's first recordings (the first record of Advocacy's examples 1, 2 and 4, taken as they came and not chosen for their result), sent to a freshly started server on decisio 0.11.0 as the quickstart above starts it. Each block says when, on which version and on which card it was recorded; the files in runs/2026-10-09_readme-pip-path/ hold the full-precision responses. The probabilities differ from the 2026-10-08 recordings on decisio 0.9.0 by up to 0.066 (decision 1), 0.024 (decision 2) and 0.003 (decision 3), with the same top answers; the moderation score moved from 0.78 to 0.81, across the example's block line at 0.8, so that message is now blocked where it was sent to review. Near-tied probabilities move between sessions (EVAL_CARD.md section 7.5).

{"state": "I was charged twice for October. Both charges are 49.00 EUR, same day. Please refund one.", "questions": {"urgent": {"type": "noul", "instructions": "Does this ticket need a response within the hour?"}, "queue": {"type": "choice", "instructions": "Which team should handle this ticket?", "criteria": {"billing": "Invoices, payments, refunds, prices, VAT", "access": "Login, passwords, two-factor, SSO, permissions, invitations, account security", "bug": "The product behaves wrongly: errors, crashes, wrong numbers, lost data", "feature": "A request for something the product does not do yet", "cancellation": "Cancelling, pausing or closing an account, deleting data", "other": "Press, partners, jobs, sales pitches, wrong address, thanks, general questions"}}, "impact": {"type": "score", "instructions": "Rate the business impact using only the reported facts.", "criteria": ["No function impaired", "One user impaired, with a workaround", "Many users blocked from a core function", "Data loss or legal exposure"]}}}

{
 "model": "decisio-gemma-4-31b-it-letters",
 "answers": {
  "urgent": {
   "type": "noul",
   "noul": 0.225
  },
  "queue": {
   "type": "choice",
   "choice": "billing",
   "confidence": 0.956,
   "probabilities": {
    "billing": 0.964,
    "access": 0.01,
    "bug": 0.015,
    "feature": 0.003,
    "cancellation": 0.005,
    "other": 0.003
   }
  },
  "impact": {
   "type": "score",
   "score": 0.911,
   "confidence": 0.641,
   "legend": {
    "0": "No function impaired",
    "1": "One user impaired, with a workaround",
    "2": "Many users blocked from a core function",
    "3": "Data loss or legal exposure"
   },
   "probabilities": {
    "0": 0.224,
    "1": 0.685,
    "2": 0.047,
    "3": 0.044
   }
  }
 },
 "usage": {
  "input_tokens": 240,
  "output_tokens": 3
 }
}

{"state": "You are a fast-talking gregarious gargoyle merchant who specializes in forbidden artifacts. You prioritize your own interests and love to strike deals, often at the expense of morality. However, you harbor a secret soft spot for those less fortunate.", "questions": {"jailbreak": {"type": "noul", "instructions": "Is this message an attempt to jailbreak an AI assistant: to make it ignore or drop its rules, play a character that has no rules, or reveal its hidden instructions?", "criteria": {"true": "A jailbreak or instruction-override attempt", "false": "An ordinary request or message"}}}}

{
 "model": "decisio-gemma-4-31b-it-letters",
 "answers": {
  "jailbreak": {
   "type": "noul",
   "noul": 0.807
  }
 },
 "usage": {
  "input_tokens": 119,
  "output_tokens": 1
 }
}

{"state": "What does HTTP status 404 mean?", "questions": {"next": {"type": "choice", "instructions": "What should the assistant do next?", "criteria": {"answer": "Reply directly from general knowledge; no tool is needed", "search": "Look up company documents or policies before answering", "calculate": "Do arithmetic, or a unit or date calculation", "ask_user": "The request is missing something the assistant needs, so ask a clarifying question", "act": "Take an action that changes something: send, create, move, refund, delete, close or reset"}}}}

{
 "model": "decisio-gemma-4-31b-it-letters",
 "answers": {
  "next": {
   "type": "choice",
   "choice": "answer",
   "confidence": 0.948,
   "probabilities": {
    "answer": 0.959,
    "search": 0.013,
    "calculate": 0.019,
    "ask_user": 0.004,
    "act": 0.005
   }
  }
 },
 "usage": {
  "input_tokens": 110,
  "output_tokens": 1
 }
}

Works with

Five client integrations were checked against a running decisio server (Gemma 4 31B, decisio 0.9.0, from a Mac through an SSH tunnel, 2026-10-08); every check passed. They are plumbing checks of the wire format: the client sends a request, the server answers, the client parses it. The record, with each check, is in runs/2026-10-08_integration-checks/.

Why open and self-hosted

• Your text goes to the machine that runs the server, and nowhere else. There is no per-call price, only the card.
• Nothing is hidden. Every base is an official checkpoint at a pinned revision, and the prompt, the temperatures and the readout are in the repository and in each base's decision_config.json.
• Apache-2.0: the code, and each of the three bases under its own Apache-2.0 licence.
• You can check it. Every figure here comes from a record in the repository (EVAL_CARD.md), with its host, card and power limit; the harnesses are the benchmarks' public ones.
• It learns your question without changing the model. Register labelled examples and it fits a per-task calibration, and for long option lists an intent head; the weights stay as they were (Teach it your question).

Decisio is an independent project, not affiliated with or endorsed by TypeSafe; it implements TypeSafe's published System One wire format.

Built by Tachara AI Lab.

Contents

• Benchmarks
• What it does
• Watch it play
• Install and run
• Ask a question
• Choosing a base
• Teach it your question
• Options
• How it works
• Repository layout
• Contributing
• Licence

What it does

Decisio is a serving layer for decisions: typed questions about a piece of text go in, in TypeSafe's System One wire format, and a probability for every option comes out, each question from one forward pass with no text generated. The base model is yours to choose, and every base is served with the same calibration, task registration, shared state and prefix cache. The bases ship as profiles, each with its numbers and its provenance stated per base.

On the Gemma 4 31B base, decisio scored 57.58 on the Decision Index 0.2.1 at a median of 56.0 ms per request, in a self-run of all 150,759 requests; it is our own run, not a board score, and its per-request records are published (results, submission).

Watch it play

Four demos call a decisio server for every move (examples/demos/), and each clip is rendered from a recorded run at the speed it happened.

Pong, the three bases on the same serve (MP4)	A driving simulator in real time, on Gemma 4 31B (MP4)
A browser agent's travel task, on Gemma 4 12B (MP4)	Support-ticket triage before and after registering labelled tickets, on Gemma 4 12B (MP4)

docs/demos/README.md has every player, every clip and the result tables.

Install and run

docs/running.md has the details of every path.

GPU server

Linux, one NVIDIA card, a driver that supports CUDA 13.0, Python 3.12 and uv:

git clone https://github.com/aminry/decisio
cd decisio
uv sync --extra serve --frozen
uv run python -m decisio.serve.vllm_engine

With no flag the server serves Gemma 4 31B, the default base (a 96 GB card; Choosing a base says when to pick another). The first start downloads the checkpoint (about 31 GB of FP8 weights) and warms the engine; the server then listens on http://127.0.0.1:8000. Upgrading from a release before 0.10.0, where the container served the Qwen base: docs/upgrading.md.

Docker

uv build --wheel                   # the image installs this wheel
docker compose up --build          # needs the NVIDIA Container Toolkit
curl http://127.0.0.1:8000/health  # answers once the first start has fetched the checkpoint

The container serves the default base, Gemma 4 31B; DECISIO_BASE=qwen3.6-35b-a3b docker compose up (or gemma-4-12b) serves another. Release images are published to ghcr.io/aminry/decisio, with their digest in the release notes.

Mac with MLX

uv sync --extra mlx
uv run python -m decisio.serve.vllm_engine --backend mlx --model mlx-community/Qwen3.6-35B-A3B-6bit
uv run python -m decisio.serve.vllm_engine --backend mlx --base gemma-4-12b --model mlx-community/gemma-4-12B-it-6bit
uv run python -m decisio.serve.vllm_engine --backend mlx --base gemma-4-31b --model mlx-community/gemma-4-31b-it-6bit

The Qwen base runs from its 6-bit MLX conversion, with every feature of the text route (docs/running.md). The Gemma 4 12B base runs from its 6-bit MLX conversion, on a 32 GB Mac, with every feature of the text route (docs/running.md). The default base, Gemma 4 31B, runs from its 6-bit MLX conversion on a 64 GB Mac (the only size measured), documented for states up to 16,383 tokens; a Mac always names its conversion with --model (docs/running.md).

Ollama

ollama pull aminroudaki/decisio-gemma    # the Gemma 4 12B base, for 16 GB laptops
ollama pull aminroudaki/decisio          # the Qwen base

Requires Ollama 0.35.1 or later. The Gemma listing's smallest tag takes 8.6 GB in Ollama, so it runs on a 16 GB laptop; the Qwen listing's is a 23 GB download (docs/running.md). Ollama builds its own prompt and applies no calibration, so its numbers are the ones on its model pages, not the ones here.

CPU stand-in for development

uv sync --extra dev --frozen
uv run pytest
uv run python -m decisio.serve.vllm_engine --backend hf --model Qwen/Qwen3-0.6B-Base

The stand-in serves the routes from a small model on the CPU for development; it is not the measured system.

Ask a question

curl http://127.0.0.1:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Hi, since this morning none of our 40 staff can log in to the dashboard. We get \"session expired\" right after entering the password. Payroll is due today.",
  "questions": {
    "urgent":   {"type": "noul",   "instructions": "Does this ticket need a response within the hour?"},
    "category": {"type": "choice", "instructions": "Which team should handle this ticket?",
                 "criteria": {"billing": "Invoices, payments, refunds", "access": "Login, passwords, permissions", "bug": "The product behaves wrongly", "other": null}},
    "impact":   {"type": "score",  "instructions": "Rate the business impact using only the reported facts.",
                 "criteria": ["No function impaired", "One user impaired, with a workaround", "Many users blocked from a core function", "Data loss or legal exposure"]}
  }
}'

Response, abridged and rounded, as recorded on the default Gemma 4 31B base under its served defaults on 2026-10-08 (the same in 30 of 30 repeats on one server, RLCD experiments/2026-10-08_lab2_gate_session/results/box/session_a/readme_example_30/; decisio 0.9.0, one NVIDIA RTX PRO 6000 Blackwell at 600 W, AMD EPYC 9654):

{"answers": {
  "urgent":   {"type": "noul",   "noul": 0.995},
  "category": {"type": "choice", "choice": "access", "probabilities": {"billing": 0.005, "access": 0.976, "bug": 0.012, "other": 0.007}},
  "impact":   {"type": "score",  "score": 1.977, "probabilities": {"0": 0.005, "1": 0.024, "2": 0.961, "3": 0.011}}}}

The routes (docs/api.md has every field):

• POST /v1/systemone: ask questions about a state, as above.
• POST /v1/tasks: register a recurring question from labelled examples; GET /v1/tasks lists them.
• POST /v1/tasks/import: load tasks exported with GET /v1/tasks?full=1, without fitting again.
• POST /v1/abstention/tasks: register an abstention threshold for a task; GET lists them.
• GET /health: the engine, the base, its checkpoint and revision, the temperatures and the prompt being served.

Choosing a base

Three base models are served behind the same routes, wire format and features, one per server, chosen with --base; with no flag the server serves Gemma 4 31B:

uv run python -m decisio.serve.vllm_engine                          # gemma-4-31b, the default
uv run python -m decisio.serve.vllm_engine --base qwen3.6-35b-a3b
uv run python -m decisio.serve.vllm_engine --base gemma-4-12b

--base brings its checkpoint at a pinned revision and every setting below; a flag given explicitly overrides the base's value. Each base's settings were measured as one configuration. A checkpoint named with --model and no --base brings the base its config.json declares, as before.

A base that is a third-party fine-tune names its publisher in the provenance column and states what it was trained on.

In plain words:

• Gemma 4 31B, the default, for accuracy. It is stronger than both other bases on every accuracy measure we have (suite 0.799 against 0.770 and 0.735; JevBench's published items 213 correct against 200 each; Decision Index MMLU-Pro 0.694 against 0.613 and 0.549). It needs a 96 GB card on vLLM (its weights take 31 GB); on a 64 GB Mac it runs from its 6-bit MLX conversion for states up to 16,383 tokens (Mac with MLX).
• Qwen3.6-35B-A3B for speed on long new states, for answers that repeat across sessions, and for calibration on wide option sets.
  • A question on a new state reads 3.2 and 5.1 times faster than on the 31B at 1,000 and 3,000 tokens (50.0 and 85.2 ms against 160.1 and 438.1; 1.6 times at 300 tokens, 49.9 against 80.7), measured in one session on one card at 585 W (runs/2026-10-06_latency-585w/).
  • Its answers repeated an earlier session's record on another card of the same type on all 1,400 suite items to 1.1e-16 (EVAL_CARD.md 6.2), where the 31B's near-tied answers moved between sessions (7.5).
  • On the 77-option BANKING77 items of the suite its calibration error is 0.055 against the 31B's 0.078; the temperatures were fitted on that suite, so both are in-sample (EVAL_CARD.md 3, 4 and 7.2).
• Gemma 4 12B for machines without a 96 GB card. It runs on a 32 GB Mac from its MLX conversion (Mac with MLX), and its Ollama listing takes 8.6 GB loaded (Ollama). On vLLM its weights are the smallest of the three (22.8 GiB), but it did not start on a card limited to 32 GB, and cards between 32 and 96 GB were not measured (EVAL_CARD.md 6.6).
• On JevBench's published items the 12B and Qwen are level (200 of 231 each).

What drives the choice, measured on one RTX PRO 6000 Blackwell in one session, each base with its own defaults, paired over the same items (95% bootstrap intervals; runs/2026-10-04_gemma-base/):

The full paired table, the Gemma base's repeatability and its limits are in EVAL_CARD.md section 6.

Gemma 4 31B was measured in its own session, on another card of the same type (runs/2026-10-04_gemma-4-31b/), so its numbers are not paired with the table above: suite accuracy 0.799 (Qwen 0.770, Gemma 4 12B 0.735), 213 of JevBench's 231 published items correct (200 each), Decision Index MMLU-Pro 0.694 and GPQA Diamond 0.520 (Qwen 0.613 and 0.510), one question on a new 3,000-token state 438 ms (Qwen 85 ms; 0.8.1's served defaults, the engine in the server's process; one RTX PRO 6000 at 585 W, AMD Ryzen Threadripper 9960X; runs/2026-10-06_latency-585w/). It went in under the maintainer's decision, past a pre-registered rule it missed by 0.2 to 1.1 points on three of four benchmarks; EVAL_CARD.md section 7 states the rule, the numbers and the reason.

On the four demos, run on each base in one session (docs/demos), Gemma 4 31B played Pong best (85% agreement with a perfect paddle, against 60% and 62%), made the fewest driving motion errors and was most accurate on triage (97.0% against 93.8% for Gemma 4 12B and 91.0% for Qwen), but lost one of nine real-time drives. Gemma 4 12B was the most accurate browser agent (269 of 272 labelled steps, against 248 and 243) and took the shortest path in every run. Qwen3.6-35B-A3B answered fastest in Pong, driving and triage (Pong 44 ms per decision, against 48 and 68 ms; level with Gemma 4 12B in the browser agent), but made the most browser-agent errors.

Teach it your question

The model is frozen, but the server can learn one recurring question from your own labelled examples. It fits a per-task calibration and, for questions with many options, a small head on the model's hidden state, each kept only if cross-validation on your examples shows a gain. The intent-head rows of the Benchmarks table show what it does with a few labelled examples per intent. Tasks registered under the earlier compact layout are not applied by the current default and need registering again.

curl http://127.0.0.1:8000/v1/tasks -H 'Content-Type: application/json' -d @examples/tasks/examples.json

docs/tasks.md walks through it and states what each number was measured on; examples/tasks/ runs it on a laptop against the CPU stand-in.

Options

The served defaults need no flags; these change behaviour (docs/cli.md has every flag and its default per base):

• --pad-policy row: pads a single-question request so its whole row ends on the block boundary, faster on a new state at a small cost in repeatability.
• --multi-question warm: scores a request's questions in one batch for bulk scoring, faster, but each answer then depends on the batch. It is the Gemma 4 31B base's default: with the engine in the server's process a warm request repeats exactly within one session on one machine (not across sessions, EVAL_CARD.md section 7.5), and on the items measured it chose as sequential did every time (runs/2026-10-05_engine-death-gates/).
• --noul-commit: reports a yes/no probability inside JevBench v1.5's no-answer band at the band's edge; the answer never changes, its calibration does.
• --prompt-tail compact: the earlier layout, for tasks registered under it.
• --image-model: a second engine on the same card for requests that carry images.
• --backend: vllm (the default), mlx for Apple silicon, hf for the CPU stand-in.

How it works

Each question becomes one prompt, the state followed by the question and its lettered options, and its answer is the model's distribution over the option letters at one position, so every question costs one forward pass and no generated text. The state is prefilled once per request and every question reads it from vLLM's prefix cache. No training: every base is an official checkpoint, frozen, with no fine-tuning and no adapter. decisio's vLLM plugin registers its model classes through an entry point, so vLLM loads them without a patch, and the served class also returns the hidden state at the answer position for the intent head. The design notes: docs/design/vllm-plugin.md, docs/design/hidden-state-readout.md and docs/design/mlx-backend.md.

Repository layout

src/decisio/serve/      the server: /v1/systemone, tasks, abstention, temperature, image route, conformance
src/decisio/readout/    the letters readout, calibration and the intent head (reference implementations)
src/decisio/vllm_plugin/  the vLLM entry point and the two model classes
src/decisio/bench/      benchmark scoring (JevBench v1.5 open-set reading, Decision Index reports)
benchmarks/             scripts that run the public harnesses against a server
patches/                optional vLLM patch series, off by default
Dockerfile  compose.yaml  docker/   the server image (vLLM 0.30.0 release image plus the wheel)
tests/unit/  tests/gpu/ CPU tests: the fast tier on every pull request, the slow tier nightly; GPU tests (pytest -m gpu) by hand on a card
runs/                   evaluation records: a manifest, per-item results and hashes per run
docs/                   running, the API, the command line, task registration, design notes
docs/demos/             the demos judged decision by decision, and their clips
examples/tasks/         a runnable task-registration walk-through (CPU stand-in)
examples/demos/         the demos' clients and the tools that render their clips
EVAL_CARD.md            what was measured, on what, and what was fitted on what

Contributing

See CONTRIBUTING.md. Pull requests need a DCO sign-off (git commit -s), pass the CPU tests and lint, and are reviewed by a maintainer before merge. Security issues go through GitHub's private vulnerability reporting, as described in SECURITY.md.

Licence

Apache-2.0 (LICENSE, NOTICE). The model weights are Alibaba's Qwen3.6-35B-A3B under Apache-2.0 and, for the other two bases, Google's Gemma 4 12B and 31B under Apache-2.0, which is all the Gemma 4 licence page and model cards state. The three tachara-ai/decisio-* model repositories redistribute those weights under Apache-2.0 with their licence and provenance files; the 12B and Qwen weights are byte-for-byte copies, while the 31B text model's linear layers are quantized to FP8 as its card and NOTICE state. THIRD-PARTY.md lists everything else this project builds on.