TEIA Cognitive Router

TEIA Cognitive Router is a deterministic, rule-based LLM router: it assigns each prompt to a Local, Hybrid, or Cloud model tier using a fixed arithmetic formula, and seals every decision with a SHA-256 audit record that can be re-verified offline.

ML-based LLM routers change behavior when their weights are retrained, so the same prompt can route differently next month and past decisions cannot be reproduced. Organizations operating under EU AI Act, GDPR, HIPAA, or SEC/FINRA requirements need to prove, after the fact and independently, why a given request went to a given model. TEIA's routing is pure arithmetic with fixed weights: the same input always produces the same decision and the same hash, and a standalone verifier can re-derive any past decision from the original prompt.

pip install teia-cognitive-router

teia-route --text "Extract all invoice numbers from this document" --output decision.json
# → routing_decision: "Local", audit_seal.sha256 attached

teia-verify --file decision.json --text "Extract all invoice numbers from this document"
# → AUDIT PASS: The routing decision is mathematically proven and unmodified.

PyPI · Paper (DOI) · License: Apache 2.0

How routing works

The router computes a Semantic Entropy Score in [0..1] from six measurable text features — no neural weights, no training data, no network calls:

The formula and weights are fixed per version; any change to weights requires a version increment, so a decision made under version X is reproducible under version X indefinitely.

Every decision is

• Reproducible — same input text → same routing decision → same SHA-256 hash
• Explainable — the routing_rationale field states which features drove the verdict
• Auditable — audit_seal.sha256 is a cryptographic commitment to the decision body; audit logs are Merkle-chained (teia-verify --verify-chain)
• Offline — stdlib-only, zero network calls, no GPU, < 5 MB memory

Measured results

• Compliance-safe mode (default): 99.6% average quality retention and 16.3% cost reduction (USD 371.25 vs 443.70/month projected) versus an all-cloud baseline, evaluated on the 80-question MT-Bench set under a fixed deterministic 3-tier quality model (Local=0.98, Hybrid=0.99, Cloud=1.00). In this mode only provably trivial prompts (entropy < 0.20) route to Local.
• Max-savings mode (opt-in): 95.2% cost reduction (USD 21.45 vs 443.70/month projected) on the same 80-question MT-Bench set — at 53.8% quality retention under the max-savings quality model (Local=0.90, Hybrid=0.95, Cloud=1.00). The trade-off is severe and stated deliberately: this mode routes 82% of MT-Bench to Local and fails the ≥95% retention target that compliance-safe mode meets. Artifact: benchmark_multidomain/quality_cost_results_max_savings.json.
• Throughput: ~10,800 routing decisions/s median, individual runs spanning 9,400–13,500, on a desktop i3-10100F with CPython 3.14 (stdlib only). The ~35% spread is turbo-clock decay across a sustained run, not routing variance — the routing distribution is bit-identical in every run. This measures route() alone (entropy plus threshold decision), excluding sealing, ledger append and gateway overhead. Method and every individual run: benchmark_multidomain/throughput_measurement.json.

Reproduce locally against public datasets:

python tests/teia_router_bench_harness.py --input tests/mt_bench_questions.json
python tests/teia_router_throughput_bench.py

Python API

from teia_cognitive_router import route_and_seal

sealed, json_str = route_and_seal("Extract all dates from this document")
print(sealed["routing_decision"])                   # "Local"
print(sealed["gpu_economics"]["delta_usd_saved"])   # 0.000440
print(sealed["audit_seal"]["sha256"])               # deterministic SHA-256

CLI tools (installed with the package)

Documentation

Design invariants

All scripts in this repository enforce:

• Idempotence: every script is safe to re-execute; same inputs produce identical outputs including SHA-256 hashes
• Determinism at boundaries: canonical JSON (sorted keys, UTF-8 without BOM), no unseeded randomness in integrity paths
• Absolute paths: no relative execution from system directories
• Entropy honesty: compression is never forced on incompressible data; every fallback verdict is declared explicitly

Version history

License

TEIA Cognitive Router is licensed under Apache 2.0. The authoritative text is LICENSE in this repository; the same license is declared in pyproject.toml and on PyPI. The benchmark artifacts in this repository are provided as-is for reproducibility of the published results.

Developed and benchmarked on modest hardware: i3-10100F · 16 GB RAM · PowerShell 7+ · Python 3.8+.