jev-lite

OpenAI-style Python client for TypeSafe Jev (System One).

Jev does not generate text. Send state plus typed questions; get structured decisions with probabilities. jev-lite is the official TypeSafe HTTP client — sync and async, same shape as the OpenAI Python SDK. jev-lite@0.0.2 added Dialogflow-style system entities; jev-lite@0.0.3 adds a privacy filter and a reranker (all three are previews, below); jev-lite@0.0.4 adds images and Box answers for Jev 1.14; jev-lite@0.0.5 adds LocalTypeSafe, the same API answered by an open model on your CPU with no API key; jev-lite@0.0.6 adds OpenAIDecisions, the same API answered by OpenAI's Decisions API; jev-lite@0.0.7 makes LocalTypeSafe read the state once per request, and measures latency by question count for every model on the board; jev-lite@0.0.8 keeps the board to a core set of 15 models.

pip install jev-lite
# or: uv add jev-lite

Python 3.11+. Set TYPESAFE_API_KEY. Docs: https://docs.typesafe.ai/

Usage

from jev_lite import Choice, Noul, Score, TypeSafe

client = TypeSafe()  # TYPESAFE_API_KEY

result = client.system_one.create(
    state="I was charged twice. Please fix this ASAP.",
    questions={
        "billing": Noul(instructions="Is this ticket about billing?"),
        "tone": Choice(
            instructions="What is the customer's tone?",
            criteria={"calm": None, "frustrated": None, "angry": None},
        ),
        "urgency": Score(
            instructions="How urgent is this ticket?",
            criteria=["can wait", "this week", "today"],
        ),
    },
)

print(result.nouls["billing"].noul)
print(result.choices["tone"].choice)
print(result.scores["urgency"].score)
print(result.elapsed_ms, result.request_id, result.usage)
print(client.models.list())  # GET /v1/models

Async is the same surface with await:

from jev_lite import AsyncTypeSafe, Noul

async with AsyncTypeSafe() as client:
    result = await client.system_one.create(
        state="I was charged twice. Please fix this ASAP.",
        questions={"billing": Noul(instructions="Is this ticket about billing?")},
    )
    models = await client.models.list()

jev-latest is an alias that moves: on 2026-10-21 it answers as jev-1.13.0, and jev-1.14.0 is opt-in. client.models.list() lists only the aliases (jev-latest, jev-preview); response.model names the version that answered. Pin a version with TypeSafe(model="jev-1.14.0") (or TYPESAFE_DEFAULT_MODEL) when answers must repeat. Prefer one system_one.create with many questions over a loop of calls. From 0.0.7 the modules are pinned to jev-1.14.0 and use the client's model. On the same day, 1.14 matched or beat 1.13 on every module below, and scored lower on hard semantic decisions (461 vs 481 of 504 on semantic-decisions@v0.0.2).

Re-running unchanged questions can replay answers from disk (pip install "jev-lite[cache]"):

client = TypeSafe(model="jev-1.14.0", cache="~/.cache/jev")  # or a diskcache.Cache

Only 200 responses are stored, keyed on the request body (which carries the model id). Pin a version: an alias such as jev-latest can move to a new model while the cache keeps replaying the old one, so a cached alias warns. Leave the cache off when you measure repeat or order stability.

Local decisions (CPU, no API key)

LocalTypeSafe answers the same system_one.create calls on this machine's CPU with vllm-sr/d3-nano (2.2B, Apache-2.0), exported to ONNX Runtime with int8 weights. No torch, no transformers, no remote code.

pip install "jev-lite[local]"   # about 40 MB download, 150 MB installed

from jev_lite import Choice, LocalTypeSafe

client = LocalTypeSafe()  # model "vllm-sr/d3-nano"
client.load()             # optional warm-up; the first run downloads 1.76 GiB
result = client.system_one.create(
    state="包裹顯示已送達，可是我家門口什麼都沒有。",
    questions={"route": Choice(criteria={"billing": "付款", "shipping": "配送"})},
)
print(result.choices["route"].choice)  # shipping

• Weights: dockhardman/d3-nano-onnx at a pinned commit, each file checked against its sha256 and cached in ~/.cache/jev-lite (cache_dir=); after that it runs offline.
• Same API: AsyncLocalTypeSafe, models.list(), cache= and the tracing spans work as for TypeSafe. Text only: an image or a Box question raises BadRequestError; a question over 8,192 tokens is refused, never truncated.
• Quality: 449/504 on the zh-Hant semantic-decisions@v0.0.2 (hard 296/336), against 481 for hosted jev-1.13.0, with the same choice as the original bf16 model on all 504. The modules below are measured on hosted Jev only.
• Speed: one question at a time on 20 Arm cores: a median of 0.5 s on short questions and 1.1 s on the bank's prompts, about 5 s at 1,100 tokens.

OpenAI Decisions

OpenAIDecisions sends the same system_one.create calls to OpenAI's Decisions API (POST /v1/decisions). It needs no extra install; only the client class changes.

from jev_lite import Choice, OpenAIDecisions

client = OpenAIDecisions()  # OPENAI_API_KEY; model "gpt-6-luna"
result = client.system_one.create(
    state="包裹顯示已送達，可是我家門口什麼都沒有。",
    questions={"route": Choice(criteria={"billing": "付款", "shipping": "配送"})},
)
print(result.choices["route"].choice)  # shipping

• Mapping: Noul → predicate (its true/false texts are appended to the instructions), Choice → choice, and Score → score levels. A question without instructions gets a short default. state that is not a string is sent as compact JSON text.
• Images: an Image in state or instructions is sent as an image part. An image in criteria and a Box question raise BadRequestError before anything is sent, as do counts the API refuses (1–400 questions, 2–255 options, 2–10 levels).
• Same API: AsyncOpenAIDecisions, cache=, typed errors, request_id, usage and the tracing spans work as for TypeSafe. models.list() returns the luna models.
• Quality: gpt-6-luna scored 442/504 on the zh-Hant semantic-decisions@v0.0.2 (hard 275/336), against 481 for Jev jev-1.13.0 and 449 for LocalTypeSafe. It changed its answer with option order on 20 of 252 rows.

Images and Box (Jev 1.14, experimental)

Put an Image anywhere a value can go: in state, instructions or criteria. The client uploads it as an asset and adds the TypeSafe-Danger-Zone-Images header. A Box answer is where a target is, as edges in fractions of the image; it needs jev-1.14.0.

from pathlib import Path
from jev_lite import Box, Image, Noul, TypeSafe

with TypeSafe(model="jev-1.14.0") as client:
    result = client.system_one.create(
        state={"photo": Image(Path("kitchen.jpg"))},  # PNG/JPEG, sides 280-1568 px
        questions={
            "has_mug": Noul(instructions="Is there a coffee mug in `photo`?"),
            "mug": Box(instructions="the coffee mug in `photo`"),
        },
    )

mug = result.boxes["mug"]
print(mug.box.left, mug.box.top, mug.box.right, mug.box.bottom, mug.confidence)

A Box always returns a box, so gate it on the Noul. An image costs about 2·⌈w/28⌉·⌈h/28⌉ input tokens. Measured limits (targets under ~40 px, counting past five, text under 7 px); the benchmark is image-questions@v0.0.1.

System entities (preview)

jev_lite.system_entities extracts the 59 Dialogflow ES system entities (@sys.date-time, @sys.unit-currency, @sys.geo-city, …) and fills typed slots, in ten languages (en and zh-Hant primary). Code proposes candidates, Jev decides (the client's model), code normalizes. Preview: the API may change within 0.0.x. Same install; no extra dependency.

from datetime import datetime
from zoneinfo import ZoneInfo

from jev_lite import TypeSafe
from jev_lite.system_entities import extract

now = datetime(2026, 10, 1, 16, 0, tzinfo=ZoneInfo("Asia/Taipei"))
with TypeSafe() as client:
    result = extract(client, "Book a table for 4 next Friday at 7pm", ["date-time", "number"], now=now)
for match in result.matches:
    print(match.entity, match.span.text, match.value)  # number 4 …; date-time … 2026-10-09 19:00

Slot filling (extract_slots), async (aextract, aextract_slots) and the value models are documented in the module's docstrings. Extraction is not a security boundary, probability is not calibrated, and th, ms and other languages outside the ten get few matches. On the held-out set system-entities-cases@v0.0.1 (301 cases, ten languages) jev-lite@0.0.5 (unchanged in 0.0.6) is 98.9% exact on accepted values with 98.9% coverage on jev-1.13.0 (jev-lite@0.0.4: 98.9% on jev-1.13.0, 99.3% on jev-1.14.0).

Privacy filter and reranker (preview)

jev_lite.privacy_filter and jev_lite.reranker ship in jev-lite@0.0.3 as previews: the APIs may change within 0.0.x. Import them as modules (from jev_lite.privacy_filter import detect, from jev_lite.reranker import rerank). Both use the client's model and are pinned to jev-1.14.0 from 0.0.7; the numbers are on jev-1.13.0, with the same-day 1.14 run beside them.

Privacy filter: private-span detection. Public multilingual gold (ai4privacy OpenPII, Gretel finance, NVIDIA Nemotron-PII), 300 test texts, exact-span precision / recall:

The default uses per-label proposers and cuts fitted on dev, and finds people with system_entities person spans plus a person detect Noul (person recall 43.9 → 71.9), with an order-stable name boundary. On overlap it scores 88.9 / 77.6; exact address recall is 0 because the gold splits an address into components while the filter masks the whole address. It fails most written robustness bars, so it is a preview. Hosted Jev sees the raw text before masking.

Reranker: batched evidence Noul over a first-stage pool (balanced batches of at most 30, independent of the caller's order), nDCG@10:

A batch score is relative to the other passages in its request, not a standalone probability. A passage that echoes the relevance criteria can be lifted into the top 10; rerank(..., check_echo=True) (off by default) re-checks the top-k and moves such passages down, at about +0.23 s.

Tracing

Off by default. Opt in and pick the destination:

from jev_lite import instrument

instrument(endpoint="http://127.0.0.1:4318")  # local Grafana LGTM

Or set OTEL_EXPORTER_OTLP_ENDPOINT and call instrument(), or install your own TracerProvider first (Logfire, etc.). Spans record prompt, result, and latency.

Upgrading from 0.0.1–0.0.5

• Pin the model. jev-latest moves to new versions, and answers and confidence shift with it. Use TypeSafe(model="jev-1.14.0") (0.0.7; it was jev-1.13.0 before), and refit any confidence cut when you move. A cached client asking for an alias warns (0.0.6).
• Noul(true=..., false=...) works from 0.0.5. Before that, the shorthand was silently ignored; use criteria=NoulCriteria(...) there. Unknown question fields warn from 0.0.5.
• Replay cache: TypeSafe(cache="dir") (0.0.3, jev-lite[cache]) replaces hand-made caches. jev_lite.eval is research code, not a stable API.
• Confidence comes with every Choice and Score answer; do not recompute it. From 0.0.6, TypeSafe, LocalTypeSafe and OpenAIDecisions all report (n·p_max − 1)/(n − 1).
• Other backends behind the same calls: LocalTypeSafe (0.0.5, CPU, no key) and OpenAIDecisions (0.0.6). A self-hosted server that speaks /v1/systemone works through TypeSafe(base_url=...).
• Modules (system_entities, privacy_filter, reranker) use the client's model from 0.0.5. From 0.0.7 they are pinned to jev-1.14.0; their tables give 1.13 and 1.14 from the same day.

Develop

uv sync --dev
uv run ruff check . && uv run pytest

The supported public API is the TypeSafe client. Optional local engines live under jev_lite.engine (pip install 'jev-lite[ml]').

Semantic decisions v0.0.2

Dataset semantic-decisions@v0.0.2, versioned separately from the package.

252 frozen zh-Hant rows in 18 families (36 easy, 48 medium, 168 hard), gold derived from visible text. This natural zh-Hant v1 qualified freeze is v0.0.2, and it is the only semantic-decision eval in this repo. Every model answers the same 504 locked requests: each row in canonical and left-rotated option order. Charts are percent correct. The families were seen during development before the freeze, so this is not unseen confirmation.

From 0.0.8 the charts and tables show a core set of 15 models: each size class (hosted, ≤1B, 2–3B, 4B, 9B, 26–35B) with its best model, plus models from well-known makers even when they score lower. Models clearly behind others of their size are no longer re-measured. The 0.0.7 README on PyPI lists every model measured (45 scored, 53 timed).

Easy and medium are regression and "insufficient information" controls. Hard carries the difference: a priority reversal counts only if both ends of a same-case pair are right in both orders. Always answering B already scores 194/336 on hard. All 22,176 first attempts (44 models × 504) are valid, with no errors. Adapters differ, so these are task-level scores, not weight comparisons. Jev jev-1.14.0 scores 20 fewer than jev-1.13.0, all on hard (293 vs 313; a repeat gave 462 in all). From 0.0.7 jev-lite pins jev-1.14.0 anyway (owner decision): pin jev-1.13.0 yourself while it is served if hard text decisions matter most. Self-host pick: d3-nano (2.2B, Apache-2.0, 449/504), which LocalTypeSafe runs (above). This bank ships in the jev-lite 0.0.2–0.0.4 sdists, so models released after 2026-10-03 may have seen it.

Earlier banks (semantic-decisions@v0.0.1, the controlled slices, r2) are retired. v0.0.1 survives only at package tag v0.0.1, and none of their scores is pooled with v0.0.2. The project redistributes no measured model's weights, except the Apache-2.0 d3-nano export that LocalTypeSafe downloads, which carries its LICENSE and NOTICE.

Latency by question count

One request, one state, N questions (1 → 255), on one machine (NVIDIA GB10, local models on its GPU, LocalTypeSafe on its CPU; hosted rows over the network). Speed depends on the hardware, so read the growth, not the milliseconds. The state is 24 joined bank policies (about 2,000 tokens); p50 of three uncached requests (each opens with a fresh request number, so no server reuses an earlier prefix). A backend that scores one question per call gets them one after another, as its client would. "cut" means a smaller N already passed 45 s or ran out of memory (local models get at most 70% of the GPU memory). Questions are numbered, so no two in a request are the same. The bar chart has every core model: what each added question costs, as a share of a one-question request, from 1 to the largest N it reached (green hosted, blue state read once, grey state read per question; 100% means N questions take N times as long). The line chart follows a few of them over N; the table gives the milliseconds.

State read is inferred from the growth: a model that reads the state once per question takes about N times as long for N questions. Hosted rows also ran 384 and 512 questions (Jev 1.14: 458 and 528 ms); Jev answered 768 questions (about 37k tokens) with HTTP 400.

"State once" rows serve the d3 checkpoints with the state read once and the questions batched on it; in d3's noncausal layers the state then no longer sees the questions. The score on semantic-decisions@v0.0.2 (/504) changes only for some sizes: nano 449 → 449, lite 353 → 342, mini 477 → 459, 27B 498 → 496 (flash: 67 → 67 on a 72-slot sample). jev-lite 0.0.7 ships only the nano one, as LocalTypeSafe.

License

Apache-2.0