SZL Holdings · Doctrine v11 · Λ = Conjecture 1 (advisory, never "green"/theorem) · canonical a-11-oy.com
SZL Router
Control before action. Evidence after.
Product · Proof · GitHub organization
Sovereign, OpenAI-compatible LLM gateway with a receipt on every answer — signed when a key is armed, else UNSIGNED-honest.
The governed control gateway in router_control.app has its own
operation and ecosystem integration contract, including
mandatory caller authentication, explicit provider allowlisting and separate
inference configuration readiness. Build it with Dockerfile.router-control.
It also supports an opt-in Ollama route fixed to the router process's loopback
address, with per-alias digest pins and browser requests bound to the selected
local provider. The control contract covers its model
identity limits and the network namespace requirement for containers.
Its SHA256 receipts are unsigned content commitments. The legacy DSSE gateway
described below is szl_router.app; the two receipt formats are distinct.
Run the local inference demo for one-command loopback inference, verified session receipts, and a hash-bound report using an installed Ollama model.
One endpoint in front of many brains — our own GPU first, then declared external candidates whose dispatch is gated — and every answer comes with a verifiable receipt (signed when a key is armed, else UNSIGNED-honest) of which model served it, on whose hardware, and at what energy/tier.
🚀 Quickstart (docker run)
docker build -t szl-router .
# Starts the API for inspection; a cloud key alone does not qualify a route:
docker run -p 8000:8000 szl-router
Point an OpenAI client at http://localhost:8000/v1. Without a qualified
sovereign upstream, the completion below fails closed with an attempt trail:
curl -i localhost:8000/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"szl-large","messages":[{"role":"user","content":"hi"}]}'
A successful completion body carries the honest x_szl_provenance block, and
the HTTP response carries an x-szl-receipt header: a base64-JSON
DSSE/ECDSA-P256 envelope you can verify independently.
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
client.chat.completions.create(model="szl-large",
messages=[{"role": "user", "content": "hi"}])
Verify a receipt (independently)
On first boot with no key configured, the server generates an ephemeral session signing key and logs its public key (honest: a per-session identity, not a persistent one). Fetch it and verify any receipt — offline, with no trust in us:
# this session's public key
curl -s localhost:8000/v1/receipt/pubkey | python -c 'import sys,json;print(json.load(sys.stdin)["public_key_pem"])' > session.pub
# grab the receipt header from a response, then verify via the CLI...
RECEIPT=$(curl -si localhost:8000/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"szl-large","messages":[{"role":"user","content":"hi"}]}' \
| tr -d '\r' | awk -F': ' 'tolower($1)=="x-szl-receipt"{print $2}')
echo "$RECEIPT" | python -m szl_router.verify --envelope - --pubkey session.pub
# ...or via the verify endpoint
curl -s localhost:8000/v1/receipt/verify -H 'content-type: application/json' \
-d "{\"envelope\":\"$RECEIPT\",\"public_key_pem\":\"$(awk '{printf "%s\\n",$0}' session.pub)\"}"
For a persistent signing identity, set SZL_RECEIPT_KEY_PEM (a PEM string) or
SZL_RECEIPT_KEY_FILE (a PEM path) from the environment. Keyless is honest:
with SZL_RECEIPT_EPHEMERAL=0 and no key, receipts are emitted UNSIGNED-honest
(signed:false) — a signature is never fabricated. Signing requires the optional
sign extra (the git-only szl-receipt library); the Docker image installs it.
Receipt fields: cost + observer + grid_context (additive)
Beyond provenance, usage and the request digest, every new receipt also carries:
cost- a per-call USD block with its evidence basis. The named paid Moonshot candidate is blocked before transport; the legacy price-table estimate and ledger do not qualify a strict concurrent spend cap. Unqualified cloud pricing/quota remains unknown (amount_usd:null), never fabricated as$0. Sovereign metal records$0.00vendor charge only, while electricity and other costs are unmetered here.observer— the observer frame the receipt was issued under: endpoint, auth mode (bearer/open), and the requested model. A receipt's verdict is honest relative to this frame — what this caller asked and how they were authenticated — never a claim about any other vantage point.grid_context(opt-in) — the observed, REPORTED grid signal at route time (carbon intensity, gCO2/kWh) from the keyless UK Carbon Intensity API (https://api.carbonintensity.org.uk/intensity), carried verbatim with itssource,observed_atandfetched_at. It follows the same honest pattern asszl-energy-attest(grid-average, never claimed "marginal"; no price, sopriceisUNAVAILABLE). It documents the grid window a run happened in — it never measures a joule (the receipt'senergystays"UNAVAILABLE") and never creates energy. Enable withSZL_RECEIPT_GRID_CONTEXT=1; the keyless fetch runs in a background thread behind a short-TTL cache (SZL_RECEIPT_GRID_TTL, default 300s) and never blocks the request path. A missing/unreachable feed yields an honest all-nullUNAVAILABLEblock — never a fabricated number.
Both fields are additive: they are simply omitted when absent, so receipts produced by older builds stay byte-identical.
Our own unified, OpenAI-compatible LLM router. One endpoint in front of many brains - our own GPU first, then configured external routes - with honest provenance on every answer.
We studied the leaders (LiteLLM proxy, OpenRouter, lm-sys RouteLLM) and then built our own, leaner version that fits our doctrine:
- Sovereign-first. Routes try our own metal (
box_gpu,nvidia_gpu) before any third-party cloud, before any paid tier. - Honest labels. Every response carries
x_szl_provenancewithserved_by,sovereign(true only for hardware we own),energy_source,tier, and the fullattemptstrail. Third-party routes are never labelled sovereign. - No secrets in the repo. All upstream keys come from the environment. Nothing secret is ever written to disk or logged.
- No half-state. A logical model either resolves to a working upstream or the call fails loud (HTTP 502) with the complete attempt trail. Unavailable providers (missing key/url) are skipped — never faked.
Logical models
| model | intent | declared order (blocked routes skipped) |
|---|---|---|
szl-large |
general large brain | box_gpu → nvidia_gpu → groq → nvidia_nim → moonshot(Kimi) |
szl-fast |
low-latency small | box_gpu → groq → nvidia_nim |
szl-coder |
coding | box_gpu → nvidia_gpu → nvidia_nim → groq |
szl-auto |
smart routing | scores the prompt, then dispatches to one of the three above |
You can also request provider:upstream_model directly. Unqualified cloud-grid
and paid Moonshot overrides are rejected before transport, including when a
credential is configured. The table preserves declaration order, not active
fallback availability.
Smart routing (szl-auto)
We studied how the leaders route by quality (lm-sys RouteLLM, OpenRouter's
auto model) and built our own, doctrine-true version. Ask for the opt-in
szl-auto model and the router scores the prompt with a deterministic,
no-LLM heuristic (heuristic-v1) — prompt length, code markers, conversation
depth, reasoning cues — then dispatches to the cheapest capable real logical
model, sovereign-first: a greeting goes to szl-fast, a hard multi-step
question to szl-large, anything with code to szl-coder.
What makes it ours: the routing decision is recorded honestly in
x_szl_provenance.routing and signed into the inference receipt — a
governed, independently-verifiable record of why a given brain answered.
It is honest by construction: the score is a routing estimate, never a quality
guarantee, it makes no extra upstream call, and it is pure — the same
prompt always routes the same way, so the receipt is reproducible. Existing
models are untouched: their provenance and receipts are byte-identical to before
(the routing block is present only for szl-auto). Tune the escalation point
with SZL_AUTO_LARGE_THRESHOLD (default 0.50).
curl localhost:8099/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"szl-auto","messages":[{"role":"user","content":"hi"}]}' -i
# -> body.x_szl_provenance.routing shows chosen_logical=szl-fast;
# the x-szl-receipt header carries the SIGNED routing decision.
Providers
| provider | tier | sovereign | key env |
|---|---|---|---|
| box_gpu | sovereign | yes | A11OY_MODEL_BASE_URL + A11OY_GPU_TOKEN |
| nvidia_gpu | sovereign | yes | NVIDIA_GPU_BASE_URL + NVIDIA_GPU_TOKEN |
| groq | unqualified-grid | no | GROQ_API_KEY |
| nvidia_nim | unqualified-grid | no | NVIDIA_NIM_API_KEY |
| zhipu | unqualified-grid | no | ZHIPU_API_KEY |
| siliconflow | unqualified-grid | no | SILICONFLOW_API_KEY |
| moonshot/Kimi | paid-grid | no | KIMI_API_KEY |
A key and URL make a provider locally configured, not qualified, free, or live. Unqualified cloud-grid candidates are skipped. The named paid Moonshot candidate is blocked before transport until exact model pricing and strict pre-call spend reservation are qualified. See provider qualification for current gates.
Run
pip install -r requirements.txt
uvicorn szl_router.app:app --host 0.0.0.0 --port 8099
curl localhost:8099/v1/chat/completions -H 'content-type: application/json' \
-d '{"model":"szl-large","messages":[{"role":"user","content":"hi"}]}'
Set SZL_ROUTER_TOKEN to require Authorization: Bearer <token> on callers. With no token set, /v1/chat/completions, /v1/embeddings and /v1/spend answer 503 (fail closed); set SZL_ROUTER_ALLOW_NO_AUTH=1 only for a deliberate open local run. Responses and /status show base_url only for built-in public provider hosts; operator-configured endpoints read "redacted" (SZL_ROUTER_EXPOSE_BASE_URL=1 restores the old output).
GET /v1/spend is a local operator endpoint exposing the legacy advisory ledger.
It admits only a loopback peer (127.0.0.0/8, ::1, or an IPv4-mapped loopback
address). When SZL_ROUTER_TOKEN is configured, that existing bearer token is
also required; a valid token does not grant remote access to this endpoint.
Missing or invalid peer addresses are denied before ledger state is read.
The check uses the ASGI server's client address and does not itself trust
Forwarded or X-Forwarded-For headers. Keep this route excluded from public
reverse-proxy routing: a local proxy can otherwise appear to be a loopback peer.
Reliability
Two layers of resilience, both honest (a real failure is always surfaced in the
attempts trail — never papered over):
-
Route failover - each logical model walks its ordered route list and returns the first eligible upstream that answers. Missing keys/URLs, unqualified cloud-grid routes, and the blocked paid Moonshot candidate are skipped and recorded in the attempt trail.
-
Same-provider transient retry — before falling through to the next route, a provider that blips transiently (HTTP
429/500/502/503/504, or a dropped connection) is retried with exponential backoff + full jitter, so concurrent callers don't retry in lockstep and stampede a recovering upstream. Permanent errors (400/401/404) are not retried — they fail through to the next route immediately. Tunable via env:env default meaning SZL_RETRY_MAX_ATTEMPTS3total tries per provider SZL_RETRY_BASE_DELAY0.25base backoff (seconds) SZL_RETRY_MAX_DELAY4.0per-sleep cap (seconds) -
Failure cooldown — after a provider fails a route (retry budget spent or a permanent error), it cools down for
SZL_COOLDOWN_SECONDS(default30,0disables) so the very next callers don't pay the same failure latency again. Honest by construction: a cooled provider is skipped only while a warm eligible fallback remains — if none remains, the cooled eligible provider is tried; every skip is recorded in theattemptstrail ascooldown-skip (...)so the receipt shows exactly why a provider was not consulted; and a single success clears the cooldown immediately.
Embedding cache identity
The legacy szl_router.app embeddings endpoint uses a process-local cache with
a default 300-second TTL and 1,024-entry cap (SZL_EMBED_CACHE_TTL and
SZL_EMBED_CACHE_MAX). Cache identity includes the resolved provider slot,
normalized endpoint, actual upstream request model, complete JSON request,
provenance labels, and SZL_EMBED_CACHE_NAMESPACE. Endpoint and credential
configuration are captured once per request so the transport uses the same
values as the cache identity. Upstream or shared router bearer-token rotation
invalidates reuse through a private per-process HMAC scope; credentials are
never placed in cache keys or responses.
Lookup follows the configured route order and skips unavailable providers.
A cached fallback cannot hide recovery of the preferred route. Consequently,
a failing preferred route is attempted before a fallback cache hit, which can
add latency. x_szl_cache.lookup_attempts records those current attempts;
x_szl_provenance.attempts retains the historical computation's provenance.
Fresh and cached responses are detached from cache storage to prevent callers
from changing later results. Inputs must be finite JSON values with string
object keys; Python objects are never stringified into cache identities, and
the Python extra argument cannot override model or input.
A hit replays an observed response, including its provider-reported model.
It does not verify an immutable weights revision behind a mutable upstream
alias. Rotate the non-secret SZL_EMBED_CACHE_NAMESPACE when changing weights
without changing endpoint/model, clear the cache with embed_cache_clear(),
or call Python embed(..., use_cache=False) to bypass caching. The namespace
is an operator assertion, not model attestation. This gateway has one optional
shared bearer token, so the cache is shared within that service scope; it does
not provide authenticated per-tenant isolation.
Arm the NVIDIA GPU
When the owned GPU is reachable on an OpenAI-compatible URL (vLLM / NIM / Ollama), set:
export NVIDIA_GPU_BASE_URL="https://<gpu-host>:<port>/v1"
export NVIDIA_GPU_TOKEN="<token>"
This legacy provider is classified as sovereign and must be used only for owned hardware. Rented or third-party hardware needs an explicitly non-sovereign provider record; a reachable URL alone does not establish ownership.
Test
python3 test_router.py
Hits whatever providers are armed via env and prints the honest provenance of each answer. Exits non-zero if nothing is wired.
The offline, deterministic suite (no network, no real keys) runs under pytest —
including test_signed_receipt.py, which mocks the upstream and proves the
signed-receipt contract end to end (signed receipt verifies, tampering fails,
keyless is UNSIGNED-honest):
pip install "szl-receipt-dsse>=0.3.3" pytest httpx
python -m pytest -q
Public status Space (governed source-of-truth)
The investor-facing status page at
SZLHOLDINGS/llm-router-live
is a static surface (no internals, no keys) and is now governed from this repo
under space/ — previously it was hand-built on HF with no GitHub source.
It is deployed whole-context (hf-space-deploy.yml, huggingface_hub.upload_folder)
because its COPY . /app Dockerfile is intentionally excluded by the org's per-file
deployer. Publication acceptance is deliberately stricter than matching bytes:
hf-space-drift-check.yml verifies the complete remote file set and every byte at
one immutable HF revision, validates its exact GitHub source binding, requires the
provider API to report RUNNING at that same runtime SHA, and requires nonce-fresh,
no-store, no-redirect HTTP 200 witnesses from /readyz and
/.well-known/szl-source.json. The latter must bind the same exact GitHub SHA
and HF revision before acceptance. PAUSED, build/start/stop/error or unknown
stages, stale revisions, missing token or endpoint configuration, malformed provider
data, redirects, auth/error pages, and non-200 readiness all fail closed.
Each run preserves hf-space-parity-evidence.json as a workflow artifact. Its
source, publication, deployment, and runtime sections are separate:
source checkout, published bytes, provider stage, and a live readiness witness are
never interchangeable claims. The verifier is read-only and does not restart, wake,
pause, publish, or otherwise mutate the Space. This is distinct from the router gateway image
(publish.yml → GHCR), which stays private and serves a different surface on :8000.
The Ouroboros loop (doctrine cross-reference)
The router is an instance of the estate's Ouroboros bounded-recursion loop. The
canonical definition is the receipt-closed kernel
szl-holdings/platform
(runLoop): bounded recursion with measurable convergence that MUST terminate on one
of four exit conditions — converged | consistent | aborted | budgetExhausted — and
emits a governance receipt for every run.
How the router embodies that same primitive (in szl_router/core.py, see Reliability above):
- Bounded. Each logical model walks an ordered, finite route list; each
eligible provider retries at most
SZL_RETRY_MAX_ATTEMPTStimes with exponential backoff + full jitter; a spent provider cools down. There is no unbounded retry — the loop always terminates, either on a working upstream or a loudHTTP 502carrying the completeattemptstrail. - Receipt-closed. Every answer carries the
x-szl-receiptenvelope; the router'sattemptstrail is the loop'sLoopTrace— the trace is the product. - Metaphor (doctrine, not math):
receipts.in ≡ receipts.out— the snake eats its own tail; each answer's provenance is fed back as an auditable input.
Honesty (Doctrine v11): Λ (the trust aggregator the kernel references) is
Conjecture 1 — advisory, never a proven theorem (unconditional uniqueness is
machine-checked FALSE; only conditional CUT-2 uniqueness is proven in
lutar-lean). This loop is a bounded,
terminating control primitive — it makes no perpetual-motion or zero-cost claim.
Explore the SZL estate: a11oy console · Receipt format spec · Lean proofs · Public proof site · Build acceptance · Telemetry · 🤗 SZLHOLDINGS
License
Apache-2.0 — see LICENSE.
Metadata
Release files for szl-router 1.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| szl_router-1.1.0.tar.gz | 161.1 kB | With provenance | Details for szl_router-1.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| szl_router-1.1.0-py3-none-any.whl | Python 3 | none | any | Details for szl_router-1.1.0-py3-none-any.whl |
Total release size: 285.0 kB
Release files / szl_router-1.1.0.tar.gz
| Download URL | szl_router-1.1.0.tar.gz |
|---|---|
| Size | 161.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9c1b9e1c5c5de1977e406074cc93f5656c107bea6528a73da768a5a8fbb94bf6
|
|
BLAKE2b-256 checksum How to use checksums |
9d9b8bbb1be4e03b36e6587c271d10f45e9d8f76d5cbf819e1bf3b080dd60452
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logRelease files / szl_router-1.1.0-py3-none-any.whl
| Download URL | szl_router-1.1.0-py3-none-any.whl |
|---|---|
| Size | 123.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
76557b2a76c315394c2b184ed5c56dfd951bab92188994450015e15ab5d5d30d
|
|
BLAKE2b-256 checksum How to use checksums |
bfd19f829579328b93b4d50cb4e4e90b9948e1e8dbdb3975d0d5133d131e2b17
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logSecurity for szl-router 1.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for szl_router-1.1.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 15, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.