mellontoken

Client for the mellontoken gateway.

import mellontoken

answer = await mellontoken.ask("Summarise this tender in one line.")
parsed = await mellontoken.ask(document_text, schema=DocExtraction)
full   = await mellontoken.ask_full(document_text, schema=DocExtraction)
full.cost_usd     # "0.00059550"
full.model        # the model that actually served it

Letting the gateway pick the model

answer = await mellontoken.ask(prompt, model="auto")
answer = await mellontoken.ask(prompt, model="auto", tier="high")   # bound the spend

full = await mellontoken.ask_full(prompt, model="auto")
full.route.model_slug    # what it picked
full.route.task          # the grid row it read
full.route.explanation   # why, in a paragraph you can argue with

One command. model="auto" makes the gateway classify the prompt on three axes (task, effort, capability), read the resulting grid cell, and serve the answer from whatever that cell names — and ask_full reports the decision on Answer.route, so routing is legible after the fact without a second call to ask about it.

Those three classifier calls are billed, per call. They cost roughly $0.005 against a completion's dollars and land on the ledger as kind=classifier rows, so the spend is visible rather than hidden. A service sending the same shape of prompt a thousand times should read full.route.model_slug once and then name that slug: the decision is a property of the workload, not of each call.

tier= bounds what the router may spend and outranks the effort classifier, because a tier is a statement about your budget rather than a property of the text. Set it per call, or once via MELLONTOKEN_TIER. It is only read on the auto path — with a named model it is logged as ignored rather than silently dropped.

A router that resolves nothing is not an error. route.model_slug is None when the capability gate rejected every candidate in the cell; ask logs the explanation and falls back to MELLONTOKEN_MODEL, which is what the gateway's own advice is — keep your own model. The one case that raises is MELLONTOKEN_MODEL=auto as well, where there is no own model to keep.

Configuration

From the environment:

or in code:

mellontoken.configure(base_url="http://gateway:8000/api/v1", api_key=key, model="gpt-5.6-luna")

From inside a container the host's gateway is host.docker.internal, not localhost.

What ask does that a bare HTTP call does not

• Sends the schema twice, as response_format and as prompt text. The request field is only honoured where the provider can enforce it; Gemini accepts a schema it will then ignore, and the field names in the prompt are what stop it inventing col_idx for col.
• Reads JSON out of a fenced reply, and wraps a bare list under the schema's single array field when that is unambiguous.
• Retries the shape, not the wire. A 401/402/403/404 is final — a key, a budget, a permission, a missing model — and raises immediately. A reply that did not parse is retried three times.
• Logs the cost of every call, which is the reason to route through the gateway at all: mellon.cost_usd is the only per-call price any provider path reports.
• Resolves model="auto" before sending anything. auto is a request for a decision, not a catalogue slug, and the completion endpoint would 404 it. Resolved once per ask, outside the retry loop: a reply that did not parse is a reason to ask the same model again, not to pay three classifiers for the same decision again.

Install

Built from this directory, which lives in the gateway's own repo so the client and the endpoint version together:

uv build            # -> dist/mellontoken-0.1.0-py3-none-any.whl