decision-toolkit

Local-first toolkit around open-weights decision models: typed choice / yes-no (noul) / score questions over a state (text or JSON), answered with a calibrated probability for every option — no generated text, no parsing.

One ONNX engine (onnxruntime), in-process, on CPU. One adapter per model contract (how a model is fed and read). Three thin surfaces over the same API: library, CLI and REST.

Co-developed by GIA-UH and Syalia S.R.L..

Documentation: https://gia-uh.github.io/decision-toolkit/. Start with Examples for the library, REST and CLI. dtk --help and every subcommand help also include runnable examples.

Installation

uv pip install decision-toolkit              # library + CLI (0.2.1 on PyPI)
uv pip install "decision-toolkit[onnx]"      # + onnxruntime, tokenizers (the model runtime)
uv pip install "decision-toolkit[server]"    # + REST server (FastAPI)

The project is managed with uv and ships a committed uv.lock. Published wheels remain compatible with pip for environments that do not use uv.

PyPI: https://pypi.org/project/decision-toolkit/0.2.1/. See docs/RELEASE.md for the Trusted Publishing flow and clean-install checks.

Three ways to use it

1. As a library (primary interface)

from decision_toolkit import DecisionModel, choice, noul, score

# download once: dtk models download gliner2.5-multi
model = DecisionModel("gliner2.5-multi")

answer = model.decide(
    state="Meeting with Acme: we agreed a 10% discount and will sign next week.",
    questions={
        "cat": choice("What kind of record is this?", {
            "meeting": "a meeting or call with topics, decisions and actions",
            "tasks": "things to do, with dates or owners",
            "contact": "a person's details: name, role, company, phone, email",
            "idea": "an idea, reflection, observation or question",
            "other": "a greeting, small talk or anything else",
        }),
        "urgent": noul("Does it need a reply today?"),
    },
)
answer["cat"].choice          # 'meeting'
answer["cat"].confidence      # 0.77
answer["urgent"].noul         # probability of "true"

verdict = model.classify(
    "Remind me: dentist next Tuesday at 9 am.",
    {"reminder": "an event with a date or time",
     "task": "a list of things to do",
     "idea": "an idea or reflection"},
    threshold=0.6,
)
verdict.label, verdict.confidence, verdict.below_threshold

The core is the superset of the surfaces. It also exposes lifecycle, usage, batch/async, capabilities and structured events:

from decision_toolkit import DecisionModel, request_context, noul

with DecisionModel("gliner2.5-multi", strict=True) as model:
    model.warmup()
    with request_context("req-42"):
        decision = model.decide("Customer wants a refund today.",
                                {"urgent": noul("Does it need a reply today?")})
    batch = model.decide_many(["first state", "second state"],
                              {"urgent": noul("Today?")})

decision["urgent"].confidence       # normalized probability
decision.usage.to_dict()            # input/output tokens + estimated flag
decision.latency_ms                 # per-decision latency
decision.request_id                 # 'req-42'
decision.truncated, decision.warnings

strict=True turns silent truncation into TruncationError. Capabilities are validated before inference (box/assets are rejected by text-only models). Custom adapters can be registered with register_adapter; local ONNX models can be opened with DecisionModel.from_onnx(...). See docs/CORE.md for the full phase 1 contract, error taxonomy, logging events and the expressiveness inventory.

2. As a CLI

dtk models list --quant                          # every variant, default, cache state
dtk models variants gliner2.5-multi --json       # real variants (quant, size, role)
dtk models download gliner2.5-multi --variant fp32  # install another quantization
dtk models use gliner2.5-multi --variant fp32       # select it persistently
dtk models current --json                        # includes the current variant
dtk decide --model gliner2.5-multi \
    --state "Meeting with Acme: 10% discount, sign next week." \
    --questions-file questions.yaml --json
dtk decide --model gliner2.5-multi \
    --input-jsonl states.jsonl --questions-file questions.yaml \
    --output-jsonl decisions.jsonl --batch-size 32
dtk classify --model gliner2.5-multi \
    --text "Dentist next Tuesday at 9" --labels "reminder,task,idea" \
    --threshold 0.6 --json
dtk config list --json
dtk models verify --all --json

dtk has consistent exit codes (0 ok, 1 runtime, 2 usage, 3 verification), --json everywhere, quiet/verbose modes and no tracebacks for expected failures. See docs/CLI.md for questions files, JSONL batch, custom --adapter/--files, config and completion.

3. As a REST service (optional extra)

uv pip install "decision-toolkit[server]"
dtk serve --model gliner2.5-multi --warmup \
  --api-key "$DECISION_TOOLKIT_API_KEY" \
  --log-format json

Endpoints: POST /v1/systemone (TypeSafe System One), POST /v1/decisions (OpenAI Decisions beta), GET /v1/models (OpenAI shape), GET /v1/decision-models (rich catalog), GET /v1/info, GET /health, GET /ready. Requests select model; variant is a documented local extension, not part of either public API. System One request validation uses the TypeSafe 422 {"detail": [...]} shape; OpenAI Decisions uses its {"error": {...}} envelope. Every response carries x-request-id. A standard System One client that sends model, state and questions can point at http://localhost:8000; image/asset and box requests are outside the current text-only subset. See docs/REST.md for the pinned contracts, limits, auth and the standard-evolution process.

The catalog is curated

The catalog is ours, not a mirror of HuggingFace. A model enters only when it is understood and verified:

1. clean commercial license;
2. an adapter whose contract we can implement from a real export;
3. a real ONNX export on the Hub;
4. measured parity against the original on a fixed sample;
5. CPU-viable latency.

Every variant points to files that exist. No placeholders, no speculative adapters (there is no generic "any ONNX" adapter).

Only m0 models are in the catalog. Researched-but-unavailable models (planned, conversion pending, broken exports) are listed in docs/ROADMAP.md and are not downloadable through the toolkit.

Per-model cards with the contract, measured evidence and known limitations: docs/models.

Variants and quantization

A quantization is a variant of a model: a real, pinned ONNX build with its own repo files, size and role. A * in the table above marks the default. dtk models list always shows every catalogued variant and what is cached:

dtk models list                      # all variants; * = default
dtk models list --quant              # add the quantization labels
dtk models variants gliner2.5-multi  # full details for one model

dtk models download fetches a variant into the model cache; it does not change the active selection. Downloading one quantization does not remove the others:

dtk models download gliner2.5-multi                     # default fp16
dtk models download gliner2.5-multi --variant fp32      # quality build
dtk models download opendecider-nano --variant fp16     # quality build
dtk models download gliner2.5-multi --variant fp32 --dry-run

Select a downloaded variant persistently, or use it for one request:

dtk models use gliner2.5-multi --variant fp32
dtk models current --json          # {"current": "...", "variant": "fp32"}
dtk decide --model gliner2.5-multi --variant fp16 --state "..." --q 'cat=choice:a|b'

From Python: DecisionModel("gliner2.5-multi", variant="fp32"). REST selects the same way with "variant": "fp32" in the request body and returns the choice under x_decision_toolkit.variant. The variant must already be downloaded. If a variant is omitted, the catalog default is used unless dtk models use selected another one.

A variant becomes default only after passing the parity gate (top-1 ≥ 99%, mean |Δp| ≤ 0.02, accuracy delta ≤ 1 pp against the fp32 build on a fixed sample) and measuring latency on CPU. Unverified conversions are not catalogued and never become a silent fallback. See Variants and quantization for the complete table, cache behavior and commands.

Operations

The server has bounded inference concurrency (--max-concurrent), an optional per-key rate limiter (--rate-limit, 429 + Retry-After), Prometheus metrics at /metrics, and optional OpenTelemetry spans (--tracing). Saturation returns 503 server_busy instead of building an unbounded queue. State and question text never become log/metric labels.

dtk serve --model gliner2.5-multi --warmup --max-concurrent 4 \
  --rate-limit 120 --rate-window 60
uv run python bench/load.py --base-url http://127.0.0.1:8000 \
  --concurrency 10 --requests 30

See docs/OPERATIONS.md for metric names, alerts, the runbook and the multi-worker caveat.

Reproducibility

Every catalog variant is pinned to a real HuggingFace commit SHA and carries a SHA256 per file, including external .onnx.data files. Downloads reject mismatches and verification treats the catalog hashes as the source of truth:

dtk models verify --all --json    # exit 3 if any file/hash/session fails

Latency/parity/calibration evidence is reproducible with the harness in bench/ and documented in docs/BENCH.md. CI runs lint, unit/contract tests and a CLI smoke from source on Python 3.12/3.13.

Wheel, sdist and Trusted Publishing releases are documented in RELEASE.md. See CHANGELOG.md, CONTRIBUTING.md, SECURITY.md and THIRD_PARTY_NOTICES.md.

Custom ONNX (at your own risk)

You can run any ONNX file with an adapter you trust — outside the curated catalog:

model = DecisionModel.from_onnx(
    "./my-model",
    adapter="gliner2",              # the layout you declare
    files={"encoder": "encoder.onnx", "classifier": "classifier.onnx",
           "tokenizer": "tokenizer.json"},
)
# no parity, no license check, no support

Third-party layouts can be registered in-process:

from decision_toolkit import register_adapter
register_adapter("my-layout", MyAdapter)

Custom models do not appear in dtk models list; the toolkit never guesses a layout. If a model deserves support, it goes through curation.

Development

uv sync --all-extras                 # locked dev, ONNX, server and tracing env
uv run pytest -q                     # fast suite (no model downloads)
uv run pytest -m integration -o addopts=""   # real model, opt-in

uv sync installs the project in editable mode and resolves every dependency from uv.lock. CI runs the same commands on Python 3.12/3.13. See CONTRIBUTING.md.

License

MIT — see LICENSE.