Skip to main content

Aegis Router v1

Sub-6ms, zero-egress LLM routing, prompt-injection defense, and tool dispatch in a single forward pass.

License Hugging Face Live Demo

Aegis Router T4 Benchmark GIF NVIDIA T4 GPU (ONNX Runtime FP16, BS=1): 5.91 ms p50 | 157.3 req/sec


Try It Locally (Zero Data Egress)

Run the full ONNX FP16 model 100% inside your browser via WebGPU/WASM—no signup, no backend, zero API calls:

👉 Launch Zero-Egress Browser Demo

Open Chrome DevTools → Network tab after loading the ~300 MB weights to verify zero network requests leave your machine. Browser WebGPU runs at ~100 ms; T4 server latency is 5.91 ms p50.


Why Aegis Router?

Production LLM gateways usually force a bad trade-off:

  • Rule-based proxies (LiteLLM/Kong): Fast, but lack semantic understanding for model tiering or adversarial prompt detection.
  • Hosted routers: Add 100–500 ms of network latency and send raw user prompts (and PII) to a third-party API.
  • LLM-as-a-Router (Haiku/GPT-4o-mini): Burns tokens and adds 400–800 ms before your actual LLM call even starts.

Aegis Router runs inside your own VPC using a single 149M-parameter ModernBERT-base encoder pass (zero tokens generated) with three multi-task heads:

  1. Calibrated Model Tiering (SLM / Mid / Frontier): Confidence threshold set at 0.85—anything below automatically escalates to Frontier so routing errors fail safe.
  2. Prompt-Injection Guardrail: Hard-blocks adversarial inputs before downstream LLM spend.
  3. 5-Class Tool Dispatch: Routes directly to DIRECT_CHAT, CODE_OPS, RAG_RETRIEVAL, STRUCTURED_SQL, or EXTERNAL_API.

Benchmarks

Evaluated on an NVIDIA T4 GPU (ONNX Runtime FP16, batch_size=1):

Benchmark Aegis Router v1
p50 Latency 5.91 ms
p95 / p99 Latency 8.44 ms / 9.08 ms
Throughput (BS=1) 157.3 req/sec
Prompt-Injection Guardrail 99.07% Acc (0.9841 F1)
5-Class Tool Dispatch 98.75% Acc (0.9878 Macro-F1)
Expected Calibration Error (ECE) 0.0572

Aegis Router T4 Throughput Benchmark Reproducible Kaggle T4 run (N=500, BS=1) — see notebooks/aegis_t4_benchmark.ipynb


Python Package & Gateway Quickstart

1. Install

# Editable install (for development)
pip install -e .

# Or install directly from GitHub
pip install git+https://github.com/devxa2602/AegisRouter.git

# Optional: GPU acceleration
pip install -e ".[gpu]"

2. Initialize Config

aegis-router init

This creates aegis_config.yaml in your current directory. Edit it to:

  • Swap models per tier — point tier 0/1/2 at any OpenAI-compatible endpoint (vLLM, Ollama, Gemini, Bedrock gateways):
    tiers:
      0:
        model: "my-local-llama"
        base_url: "http://localhost:8080/v1/chat/completions"
        api_key_env: "LOCAL_API_KEY"
    
  • Set custom pricing / zero-cost local models:
    pricing:
      overrides:
        my-local-llama:
          input_cost_per_1m: 0.0
          output_cost_per_1m: 0.0
    
  • Air-gapped VPC? Set pricing.offline_mode: true — Aegis falls back to its local cache:
    pricing:
      offline_mode: true
      cache_path: "~/.cache/aegis_router/model_prices.json"
    

3. Start the Gateway

aegis-router serve --config aegis_config.yaml --host 0.0.0.0 --port 8000

Then swap one line in your OpenAI SDK client:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",   # ← point here
    api_key="your-api-key",
)

resp = client.chat.completions.create(
    model="ignored-by-aegis",  # Aegis routes to the right tier automatically
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(resp.choices[0].message.content)

4. ROI Reporting

CLI:

aegis-router report --days 7 --db aegis_telemetry.db

API:

curl http://localhost:8000/v1/roi?days=7

Both return: total requests, blocked injections, fail-safe escalations, p50/p95/p99 Aegis latency, actual spend vs. all-frontier counterfactual, net savings ($ and %), breakdown by tier & tool, and count of UNPRICED requests.


Quickstart (Direct ONNX Inference)

1. Download Weights

Weights are hosted on Hugging Face: AkshaXA/aegis-router-v1

pip install onnxruntime-gpu transformers huggingface_hub

2. Run Inference

from huggingface_hub import hf_hub_download
import onnxruntime as ort
from transformers import AutoTokenizer
import numpy as np

model_path = hf_hub_download("AkshaXA/aegis-router-v1", "aegis_router_v1_fp16.onnx")
tokenizer = AutoTokenizer.from_pretrained("AkshaXA/aegis-router-v1")
session = ort.InferenceSession(model_path, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])

inputs = tokenizer("Write a SQL query to find monthly active users.", return_tensors="np", truncation=True)
outputs = session.run(None, {
    "input_ids": inputs["input_ids"].astype(np.int64),
    "attention_mask": inputs["attention_mask"].astype(np.int64),
})

Roadmap & Enterprise

Open-Source (Apache 2.0):

  • Aegis Router v1 weights (149M), ONNX FP16 graph, basic FastAPI proxy, and WebGPU demo.
  • Coming in v2 (Aegis 300M): Custom 3:1 Mamba + FlashAttention-2 hybrid backbone for linear $O(N)$ routing and indirect prompt-injection defense across 64K–128K multi-turn agent trajectories.
  • Enterprise Control Plane: Custom LoRA heads for proprietary tool schemas, RBAC, and SOC2 audit logging. Interested in a free in-VPC design partnership? Reach out at akshayrawate@gmail.com.

License: Apache 2.0 — see LICENSE for details.

Metadata

Release files for aegis-ai-router 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aegis-ai-router 1.0.0
File Size Uploaded Provenance Actions
aegis_ai_router-1.0.0.tar.gz 290.2 kB Without provenance Details for aegis_ai_router-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aegis-ai-router 1.0.0
File Interpreter ABI Platform Actions
aegis_ai_router-1.0.0-py3-none-any.whl Python 3 none any Details for aegis_ai_router-1.0.0-py3-none-any.whl

Total release size: 311.8 kB

Release files / aegis_ai_router-1.0.0.tar.gz

Download URL aegis_ai_router-1.0.0.tar.gz
Size 290.2 kB
Tags Source
SHA-256 checksum
How to use checksums
63f4cbcc704edd1dc75dd31b689cb61e7214fa249a1133cc0e200917911eb2ca
BLAKE2b-256 checksum
How to use checksums
39e813097389f3ce4dcb86802eee0f4bfdb1809cdbb62d17a656e7176fa412ba
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.0

Release files / aegis_ai_router-1.0.0-py3-none-any.whl

Download URL aegis_ai_router-1.0.0-py3-none-any.whl
Size 21.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c9a61b0f426722c758802b09c47af4ff88e2800b6ea5cc83f6502c216e14c1e3
BLAKE2b-256 checksum
How to use checksums
d633f59313b43a309ad6577118bfffd020344a3c22b8463df4c0b08e9e540839
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.0

Security for aegis-ai-router 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.0.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page