mellontoken

Client for the mellontoken gateway.

import mellontoken

answer = await mellontoken.ask("Summarise this tender in one line.")
parsed = await mellontoken.ask(document_text, schema=DocExtraction)
full   = await mellontoken.ask_full(document_text, schema=DocExtraction)
full.cost_usd     # "0.00059550"
full.model        # the model that actually served it

Letting the gateway pick the model

answer = await mellontoken.ask(prompt, model="auto")
answer = await mellontoken.ask(prompt, model="auto", tier="high")   # bound the spend

full = await mellontoken.ask_full(prompt, model="auto")
full.route.model_slug    # what it picked
full.route.task          # the grid row it read
full.route.explanation   # why, in a paragraph you can argue with

One command, and one request. model="auto" goes to the gateway as the model name; the gateway classifies the prompt on three axes (task, effort, capability), reads the resulting grid cell, and serves the answer from whatever that cell names — all inside the same call. ask_full reports the decision on Answer.route, so routing is legible after the fact without a second call to ask about it.

Those three classifier calls are billed, per call. They cost roughly $0.005 against a completion's dollars and land on the ledger as kind=classifier rows, so the spend is visible rather than hidden. A service sending the same shape of prompt a thousand times should read full.route.model_slug once and then name that slug: the decision is a property of the workload, not of each call.

tier= bounds what the router may spend and outranks the effort classifier, because a tier is a statement about your budget rather than a property of the text. Set it per call, or once via MELLONTOKEN_TIER. It is only read on the auto path — with a named model it is logged as ignored rather than silently dropped.

A router that resolves nothing is not an error. When the capability gate rejects every candidate in the cell the gateway answers 422 with the reason; ask logs it and retries against MELLONTOKEN_MODEL, which is what that error advises — keep your own model, so auto is never less reliable than naming a slug. The one case that raises is MELLONTOKEN_MODEL=auto as well, where there is no own model to keep.

Configuration

From the environment:

or in code:

mellontoken.configure(base_url="http://gateway:8000/api/v1", api_key=key, model="gpt-5.6-luna")

MELLONTOKEN_API_KEY is the only variable most callers set. The default base URL is the dev gateway, which resolves on the internal network only — from outside it, set MELLONTOKEN_BASE_URL yourself. Running the stack locally, that is http://localhost:8000/api/v1, or http://host.docker.internal:8000/api/v1 from inside a container, where localhost is the container rather than the host.

What ask does that a bare HTTP call does not

• Sends the schema twice, as response_format and as prompt text. The request field is only honoured where the provider can enforce it; Gemini accepts a schema it will then ignore, and the field names in the prompt are what stop it inventing col_idx for col.
• Reads JSON out of a fenced reply, and wraps a bare list under the schema's single array field when that is unambiguous.
• Retries the shape, not the wire. A 401/402/403/404 is final — a key, a budget, a permission, a missing model — and raises immediately. A reply that did not parse is retried three times.
• Logs the cost of every call, which is the reason to route through the gateway at all: mellon.cost_usd is the only per-call price any provider path reports.
• Sends model="auto" straight through, and lets the gateway route it. This client used to resolve it first against /api/auto/preview and then send the slug that named — two round trips for one logical request. That endpoint still exists, for callers that want a decision without a completion; this path no longer uses it. Routing happens once per ask: the first reply names the model it chose and the shape-retries are pinned to it, so a reply that did not parse is a reason to ask the same model again rather than to pay three classifiers a second time.

Install

Built from this directory, which lives in the gateway's own repo so the client and the endpoint version together:

uv build            # -> dist/mellontoken-0.1.0-py3-none-any.whl