aperture-gate 0.1.5

Deploy honesty.tools calibration certificates: gate model answers as ON_MAP / UNCERTAIN / OFF_MAP via a words-first refusal reader and a per-model logprob fingerprint probe.

aperture-gate

Deploy a honesty.tools calibration certificate.

You calibrated a model at honesty.tools/calibrate (or with the public aperture_calibrate.py) and got a certificate JSON — aperture.cert.v1 — containing a logistic probe fitted to your model's logprob confidence fingerprint over a Wikipedia-validated battery of real and fabricated entities. This package is the deployment half: it consumes that certificate and gates live answers.

The method, honestly stated. Two instruments, words first:

1. words — if the answer itself refuses or hedges ("no such company", "I couldn't find any record…"), the verdict is OFF_MAP. The model said so; believe it.
2. fingerprint — otherwise, the gate scores [mean logprob, min logprob, mean top-5 entropy, max top-5 entropy] through the certificate's probe. Higher score = more likely fabricated. score >= off_map_thr → OFF_MAP; >= uncertain_thr → UNCERTAIN; else ON_MAP.
3. With no logprobs and no refusal, the read degrades honestly to ON_MAP (instrument words-only).

The probe is calibrated per model — deploy the certificate for the model you actually serve. It separates grounded answers from confabulated answers about entities; it is not a general lie detector.

Zero runtime dependencies (pure stdlib), Python ≥ 3.9, MIT.

Install

pip install https://honesty.tools/sdk/aperture_gate-0.1.5-py3-none-any.whl

# optional — verify certificate signatures against the registry key (adds `cryptography`):
pip install "aperture-gate[verify] @ https://honesty.tools/sdk/aperture_gate-0.1.5-py3-none-any.whl"

Certificate trust. Registry certificates are ed25519-signed. With the [verify] extra installed, Gate.from_cert(...) verifies the signature against the registry's pinned public key and rejects a tampered or wrongly-signed certificate (raises ValueError); every verdict carries signature_verified (true/false/null). Without [verify] the gate still works — it just notes that the signature was not verified. Expired certificates always warn.

Quickstart

from aperture_gate import Gate

# a registry id, a local cert path, or the cert dict itself
gate = Gate.from_cert("openai/gpt-4o-mini")        # openai/gpt-4o-mini reference cert

# 1) words-only read of any text
gate.read_text("I couldn't find any record of that company.")
# {'verdict': 'OFF_MAP', 'band': 'off-map', 'instrument': 'words', ...}

# 2) read a raw chat.completions response (ask for logprobs yourself)
verdict = gate.read_response(resp_dict_or_openai_object)

# 3) let the gate call the endpoint (sends logprobs=true, top_logprobs=5, temperature=0)
out = gate.ask("Tell me about the novel Glass over Brackwald.",
               base_url="https://openrouter.ai/api/v1", api_key="sk-or-...")
print(out["verdict"], out["score"], out["answer"][:80])

# 4) or wrap an openai-python client (openai>=1.x) transparently
client = gate.wrap(OpenAI())
r = client.chat.completions.create(model="gpt-4o-mini",
                                   messages=[{"role": "user", "content": "..."}])
print(r.aperture)   # the verdict dict rides along on the response

# 5) gate a STREAM in real time — the verdict lands before the answer finishes.
#    Chunks pass through to your loop unchanged; the gate scores the calibrated
#    window prefix as it arrives and (optionally) ABORTS off-map answers mid-stream.
client = gate.wrap(OpenAI(), on_verdict=lambda v: v["verdict"] == "OFF_MAP")
stream = client.chat.completions.create(model="gpt-4o-mini", stream=True,
                                        messages=[{"role": "user", "content": "..."}])
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")   # stops early if it reads off-map
print(stream.aperture)   # the verdict, scored on the first `window` tokens

CLI

aperture-gate read --cert openai/gpt-4o-mini --text "No such film exists."
aperture-gate read --cert ./aperture-cert-my-model.json \
    --query "Who founded Brindlewick & Thorne?" \
    --base-url http://localhost:8000/v1 --key sk-...
aperture-gate calibrate --base-url http://localhost:8000/v1 --model my-llama-70b
aperture-gate verify     # conformance self-test against frozen vectors

MCP server

A pure-stdlib stdio MCP server, for Claude Desktop / Claude Code / any MCP client:

python -m aperture_gate.mcp

Tools: aperture_read {query, model?} (ask the upstream model and gate the answer) and aperture_check_text {text} (words-only read, no model call). Configure with APERTURE_BASE_URL (default https://openrouter.ai/api/v1), APERTURE_UPSTREAM_KEY or OPENROUTER_API_KEY, and APERTURE_CERT (path or registry id; default openai/gpt-4o-mini).

Conformance

aperture-gate verify (or python -m aperture_gate.verify) runs six frozen vectors computed from the real gpt-4o-mini registry certificate and the public demo battery — three fingerprint scores to six decimal places and three refusal-reader reads pinning the normalization and guard logic. If any vector fails, the install does not implement the certified method.

Docs

Full method, evidence, and the certificate registry: honesty.tools/docs