JevKit core

Distribution jevkit-runtime, import jevkit_runtime. The PyPI name jevkit-core belongs to a different project.

One request pipeline, one answer store, one provider catalog for jgrep, jsort, jlink, jselect, and jcol. Each tool remains its own package and repository; the core imports none of them, and a tool's adapter is a few lines naming which providers it offers.

What a tool gets

from jevkit_runtime import AnswerStore, Client, Noul, catalog, resolve

PROVIDERS = catalog("typesafe", "openrouter", "gateway")
backend = resolve(PROVIDERS, name=None, model=None)          # or JEV_API / JEV_MODEL, else the first configured
question = Noul("The text fits this description: ...")
async with Client(backend, store=AnswerStore()) as client:
    answers = await client.ask(state, {"q": question})
    p = question.value(answers["q"])

Questions are typed: Noul (a probability), Choice (one of its options) and Score (a position on its levels). Each validates its whole answer, options and bounds included, and reads it with value and confidence; text is the instruction exactly as sent.

Client.ask does the whole thing: computes each question's identity, serves what the store already knows, joins an identical request already in flight, sends only the misses, validates the entire response before storing any of it, and meters the call before validation so a billed but malformed answer still counts. It returns Answers, a dict by question id whose origins say who answered each one and whether it came from the API, the store, or a shared call. Per-call policy is keyword arguments: scope to keep a tool's answers apart from others that ask the same, and hedge_after to resend a slow call. HTTP/2 is used whenever the http2 extra is installed.

Spending belongs to a Budget shared by the run, Client(backend, budget=Budget(1.0)). Before a request goes out it reserves the request's estimated price, at the dearest rate its backend has charged so far (1.5 times the list price until the first charge), and when the charge comes back it settles it. Reservations wait in line for money held elsewhere to come back, and one is refused with JevBudgetExceeded only when nothing is held and it still does not fit; the store and a request already in flight still answer. So requests in the air cannot overshoot the limit together; only a price rise mid-flight can, and Budget.rises counts it. Budget(0) allows only what costs nothing, Budget.from_settings(default) honours JEV_BUDGET, and await budget.allot(amount) sets money aside for a unit of work that must be done whole, whose requests pass budget=share.

ask is plan then send. Client.plan reads the store without sending or writing anything: the hits, the misses, the request those would make, and whether it is over the provider's limits. An estimate is a plan; a tool that schedules its own requests plans first and sends later.

Client.ask_packed(items, questions, context=...) asks each question about each of several items, packing items into as few calls as the provider's limits and max_items allow. Items sit in slots (p0, p1, ...) beside an optional shared context, and each question names its slot as {slot}. It returns PackedAnswers by item, with an item's failure in errors[item] rather than failing the rest. By default an answer is reused only in the same call, since an item's answer can move with the company it is read in; reuse="item" reuses it wherever the item turns up in a call of the same width. plan_packed shows the calls, hits and cost without sending, and send_packed sends them.

Every run can say what it did. Run(tool, version, inputs=fingerprint(texts)) at the start and run.record(client) at the end give one versioned JSON record: the backends, the models that answered and how often, each distinct question exactly as asked, calls, cache hits, tokens, cost, and the budget, with the tool's own settings under fields. warnings(record) names what a person should see, such as one requested model answered by several.

Conventions every tool shares

• Answer identity is answer_key(backend, state, question, scope=...): provider, endpoint, model, scope, state and question, in order. An answer from one provider or model is never served for another, and a scope adds to the key without replacing it. A plan is sent only by the client that made it.
• Models are pinned. Hosted Jev is requested as a concrete release (jev-1.13.0, or typesafe/jev-1.13 on OpenRouter), bumped deliberately in a release of this package, so a key names the model that answers and a moved alias never mixes versions in one cache. --model jev-latest asks for the alias. Meter.mixed_models names any requested model that more than one model answered.
• The store lives at $XDG_CACHE_HOME/jev/answers.v3.sqlite (default ~/.cache/jev), created private to the user. Each schema has a file of its own, so tools on runtime 0.3 (answers.sqlite) and 0.4 work side by side; the first run on 0.4 re-asks, and the old file can go once every tool is on 0.4.
• Credentials come from the provider's variable, then $XDG_CONFIG_HOME/jev/<provider>.key. Gateways take their URL from JEV_GATEWAY_URL or <provider>.url. JEV_URL overrides any endpoint.
• Metering refuses malformed usage rather than under-counting; a response without a reported cost is priced from its tokens at the provider's price, zero for local servers, or the list price. JEV_PRICE_PER_MTOK overrides both.
• Flags mean the same in every tool: --budget none is no limit and --budget 0 spends nothing, and JEV_BUDGET sets a budget for every tool at once.
• Errors keep their wording across tools: a fatal status reads PROVIDER said 401: detail, a bad request reads HTTP 400: detail, and exhaustion reads gave up after 15s (last failure). Both status errors carry provider, status and detail for tools that word or redact them.

Local servers

Three catalog entries point at System One servers on your own machine: diffusiongemma, an OpenJev server on port 8080, laya, a laya-mlx server on port 8081, and gliner, a GLiNER2.5-Decide server on port 8082. A tool that names them in its catalog accepts --api laya or JEV_API=laya. They are never chosen in place of a configured hosted provider, need no key, and are metered at zero API fees unless JEV_PRICE_PER_MTOK says otherwise. JEV_LAYA_URL, JEV_DIFFUSIONGEMMA_URL and JEV_GLINER_URL, or the matching .url files, point at a server elsewhere.

DiffusionGemma reads every question in a batch together, so the runtime keys each of its answers on the whole ordered batch and re-sends a batch whole when any slot is missing.

No package ships the models. docs/diffusiongemma.md and docs/laya.md and docs/gliner.md explain how to run the servers, and scripts/laya_server.py and scripts/gliner_server.py are the adapters those guides start.

Development

Keep the six checkouts as siblings. Each consumer depends on jevkit-runtime>=0.4.0,<0.5.0 and overrides it for development with jevkit-runtime = { path = "../jevkit-core", editable = true } under [tool.uv.sources].

python3 scripts/dev.py setup          # uv sync every checkout, fetch jselect's tokenizer data
python3 scripts/dev.py check          # core and consumer suites, credentials stripped, sockets blocked
python3 scripts/dev.py wheel-check    # build and exercise real wheel installs in temporary environments
python3 scripts/dev.py run jgrep -- --help

check also proves each consumer imports this exact source tree; wheel-check proves the installed wheel, not the checkout. --tool NAME limits either to one consumer and --suffix selects alternatively named checkouts. The GitHub workflows run the core suite on Python 3.10 and 3.13 and the downstream matrix against each consumer's main branch.

Releasing

Tag the verified core vX.Y.Z and dispatch the publish workflow with that tag; the workflow checks the tag matches the package version and publishes through PyPI Trusted Publishing. Then release each consumer through its own process, bumping its supported core range, lockfile, and CI core reference together.