typevet

Kind: landing page (the project overview; the one page that mixes kinds).

typevet is a Python library that asks a model typed questions and returns typed answers. The three question types are Noul (yes or no), Choice (one label) and Score (one rubric level). typevet computes each answer from the model's next-token probabilities, read before sampling. typevet also returns JSON objects that pass a JSON Schema you supply, or it raises an error. The receipts cover Gemma 4 31B on llama.cpp for local work and on vLLM for hosting.

Read the documentation at https://alberto-codes.github.io/typevet/.

Status

• typevet is pre-1.0. The package version is 0.2.0.
• Install typevet from PyPI: pip install typevet or uv add typevet. See Install typevet.
• typevet requires Python 3.12 or later.
• Each tested pin has one receipt. The receipt gives the full pin and its limits.

Performance: on one H100 at concurrency level 64, 480 Banking77 records took 12.1 s at 39.6 records/s. That run had 0 errors. Banking77 calibration passed; DIFrauD SMS failed parity (ECE 0.158 against 0.10). One run, one pod, one pin. See Performance on one H100 and Serve Gemma 4 31B on a rented H100. A valid structure does not prove accuracy or calibration. The receipts are small samples.

See it run

https://github.com/user-attachments/assets/4d6dd426-a3a1-4e3e-9e93-17e3513fe333

This recording is from 2026-10-02: model gemma-4-31b-kv9-q4km-mm, llama.cpp build b11347-5fc4f3c8c, one NVIDIA GeForce RTX 4090. It shows four live cases, each with its receipt. Three image cases return a Choice label. One text case asks a Noul, a Choice and a Score question. The type system refuses the GIF upload before typevet calls the router. The recording shows typed answers and receipts. It does not prove accuracy. To run the live demo, follow Stand up the live demo.

Quickstart

Get one offline typed judgment from a scripted fake. This step needs no model.

pip install typevet
python -c "
from typevet.domain import Noul
from typevet.runtime import ScoringJudgmentAdapter
from typevet.testing import ScriptedScoringFake
fake = ScriptedScoringFake(logprobs={'True': -0.2, 'False': -1.0})
port = ScoringJudgmentAdapter(fake, tokenize_content=lambda t: (ord(t[0]),))
r = port.judge('text', {'q': Noul(instructions='Ok?', criteria={'true': 'Y', 'false': 'N'})}, 'fake')
print('noul', r.nouls['q'].noul)
"

The command prints the probability of yes, near 0.69. The offline tutorial explains each step. Then connect a model server:

• To host typevet, follow Serve typevet on vLLM.
• To run typevet locally, follow Run Gemma 4 on llama.cpp.
• To call typevet from code, follow Call typevet from Python.

Learn more

• How typevet works with Gemma 4 explains the scoring path, the two backends and the receipts.
• Gemma 4 multimodal judgments explains how images reach each backend, and the limits.
• Native typed judgments states the scope and the limitations.
• The documentation index lists every page and its kind.

TypeLLM is a research reference for the decision model. It is not a runtime dependency.

For contributors

Read CLAUDE.md first. It states the gates, the issue workflow and the rules for agents and people.

uv sync
uv run pre-commit install -t pre-commit -t pre-push -t commit-msg
uv run pytest -q

The default test run skips live tests. Pull requests and pushes to main run the hook stages in the CI workflow. The writing system and the commit rules apply to every change.