Aegis Router v1
Sub-6ms, zero-egress LLM routing, prompt-injection defense, and tool dispatch in a single forward pass.
NVIDIA T4 GPU (ONNX Runtime FP16, BS=1): 5.91 ms p50 | 157.3 req/sec
Try It Locally (Zero Data Egress)
Run the full ONNX FP16 model 100% inside your browser via WebGPU/WASM—no signup, no backend, zero API calls:
👉 Launch Zero-Egress Browser Demo
Open Chrome DevTools → Network tab after loading the ~300 MB weights to verify zero network requests leave your machine. Browser WebGPU runs at ~100 ms; T4 server latency is 5.91 ms p50.
Why Aegis Router?
Production LLM gateways usually force a bad trade-off:
- Rule-based proxies (LiteLLM/Kong): Fast, but lack semantic understanding for model tiering or adversarial prompt detection.
- Hosted routers: Add 100–500 ms of network latency and send raw user prompts (and PII) to a third-party API.
- LLM-as-a-Router (Haiku/GPT-4o-mini): Burns tokens and adds 400–800 ms before your actual LLM call even starts.
Aegis Router runs inside your own VPC using a single 149M-parameter ModernBERT-base encoder pass (zero tokens generated) with three multi-task heads:
- Calibrated Model Tiering (
SLM/Mid/Frontier): Confidence threshold set at0.85—anything below automatically escalates to Frontier so routing errors fail safe. - Prompt-Injection Guardrail: Hard-blocks adversarial inputs before downstream LLM spend.
- 5-Class Tool Dispatch: Routes directly to
DIRECT_CHAT,CODE_OPS,RAG_RETRIEVAL,STRUCTURED_SQL, orEXTERNAL_API.
Benchmarks
Evaluated on an NVIDIA T4 GPU (ONNX Runtime FP16, batch_size=1):
| Benchmark | Aegis Router v1 |
|---|---|
| p50 Latency | 5.91 ms |
| p95 / p99 Latency | 8.44 ms / 9.08 ms |
| Throughput (BS=1) | 157.3 req/sec |
| Prompt-Injection Guardrail | 99.07% Acc (0.9841 F1) |
| 5-Class Tool Dispatch | 98.75% Acc (0.9878 Macro-F1) |
| Expected Calibration Error (ECE) | 0.0572 |
Reproducible Kaggle T4 run (
N=500, BS=1) — see notebooks/aegis_t4_benchmark.ipynb
Python Package & Gateway Quickstart
1. Install
# Editable install (for development)
pip install -e .
# Or install directly from GitHub
pip install git+https://github.com/devxa2602/AegisRouter.git
# Optional: GPU acceleration
pip install -e ".[gpu]"
2. Initialize Config
aegis-router init
This creates aegis_config.yaml in your current directory. Edit it to:
- Swap models per tier — point tier 0/1/2 at any OpenAI-compatible endpoint (vLLM, Ollama, Gemini, Bedrock gateways):
tiers: 0: model: "my-local-llama" base_url: "http://localhost:8080/v1/chat/completions" api_key_env: "LOCAL_API_KEY"
- Set custom pricing / zero-cost local models:
pricing: overrides: my-local-llama: input_cost_per_1m: 0.0 output_cost_per_1m: 0.0
- Air-gapped VPC? Set
pricing.offline_mode: true— Aegis falls back to its local cache:pricing: offline_mode: true cache_path: "~/.cache/aegis_router/model_prices.json"
3. Start the Gateway
aegis-router serve --config aegis_config.yaml --host 0.0.0.0 --port 8000
Then swap one line in your OpenAI SDK client:
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1", # ← point here
api_key="your-api-key",
)
resp = client.chat.completions.create(
model="ignored-by-aegis", # Aegis routes to the right tier automatically
messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(resp.choices[0].message.content)
4. ROI Reporting
CLI:
aegis-router report --days 7 --db aegis_telemetry.db
API:
curl http://localhost:8000/v1/roi?days=7
Both return: total requests, blocked injections, fail-safe escalations, p50/p95/p99 Aegis latency, actual spend vs. all-frontier counterfactual, net savings ($ and %), breakdown by tier & tool, and count of UNPRICED requests.
Quickstart (Direct ONNX Inference)
1. Download Weights
Weights are hosted on Hugging Face: AkshaXA/aegis-router-v1
pip install onnxruntime-gpu transformers huggingface_hub
2. Run Inference
from huggingface_hub import hf_hub_download
import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np
model_path = hf_hub_download("AkshaXA/aegis-router-v1", "aegis_router_v1_fp16.onnx")
tokenizer = AutoTokenizer.from_pretrained("AkshaXA/aegis-router-v1")
session = ort.InferenceSession(model_path, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])
inputs = tokenizer("Write a SQL query to find monthly active users.", return_tensors="np", truncation=True)
outputs = session.run(None, {
"input_ids": inputs["input_ids"].astype(np.int64),
"attention_mask": inputs["attention_mask"].astype(np.int64),
})
Roadmap & Enterprise
Open-Source (Apache 2.0):
- Aegis Router v1 weights (149M), ONNX FP16 graph, basic FastAPI proxy, and WebGPU demo.
- Coming in v2 (Aegis 300M): Custom 3:1 Mamba + FlashAttention-2 hybrid backbone for linear $O(N)$ routing and indirect prompt-injection defense across 64K–128K multi-turn agent trajectories.
- Enterprise Control Plane: Custom LoRA heads for proprietary tool schemas, RBAC, and SOC2 audit logging. Interested in a free in-VPC design partnership? Reach out at akshayrawate@gmail.com.
License: Apache 2.0 — see LICENSE for details.
Metadata
Release files for aegis-ai-router 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| aegis_ai_router-1.0.0.tar.gz | 290.2 kB | Without provenance | Details for aegis_ai_router-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| aegis_ai_router-1.0.0-py3-none-any.whl | Python 3 | none | any | Details for aegis_ai_router-1.0.0-py3-none-any.whl |
Total release size: 311.8 kB
Release files / aegis_ai_router-1.0.0.tar.gz
| Download URL | aegis_ai_router-1.0.0.tar.gz |
|---|---|
| Size | 290.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
63f4cbcc704edd1dc75dd31b689cb61e7214fa249a1133cc0e200917911eb2ca
|
|
BLAKE2b-256 checksum How to use checksums |
39e813097389f3ce4dcb86802eee0f4bfdb1809cdbb62d17a656e7176fa412ba
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.0
|
Release files / aegis_ai_router-1.0.0-py3-none-any.whl
| Download URL | aegis_ai_router-1.0.0-py3-none-any.whl |
|---|---|
| Size | 21.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c9a61b0f426722c758802b09c47af4ff88e2800b6ea5cc83f6502c216e14c1e3
|
|
BLAKE2b-256 checksum How to use checksums |
d633f59313b43a309ad6577118bfffd020344a3c22b8463df4c0b08e9e540839
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.0
|
Security for aegis-ai-router 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.