Ignis Router

Intelligent LLM Routing Library for Python

Automatically selects the best language model for every query using ML routers, rule-based intent detection, weighted scoring, and provider fallback.

What is Ignis Router?

Ignis Router is a production-ready Python package that sits between your application and LLM providers. It uses machine learning to predict the optimal model for each query, rule-based intent detection as an intelligent fallback, and automatic provider switching when API keys are unavailable.

Your App → Ignis Router → Best LLM (OpenAI / Anthropic / Gemini) → Response

Why use it?

• Cost savings — Routes simple queries to cheaper models, complex ones to premium models
• Quality optimization — ML routers trained on 50k+ examples learn which model performs best for which query type
• Zero downtime — Automatic fallback when a provider is unavailable
• Full observability — Every routing decision is logged with correlation IDs

Architecture

User Query: "Write a Python API with authentication"
    │
    ▼
┌─ Intent Detection ────────────────────────────┐
│  Semantic ML → confidence 0.92 → code_gen     │
└───────────────────────────────────────────────┘
    │
    ▼
┌─ ML Router (SVM) ────────────────────────────┐
│  Predicts: qwen2.5-7b-instruct               │
└───────────────────────────────────────────────┘
    │
    ▼
┌─ Provider Check ─────────────────────────────┐
│  qwen2.5 → No API key → Fallback to OpenAI  │
└───────────────────────────────────────────────┘
    │
    ▼
┌─ LLM Call ───────────────────────────────────┐
│  gpt-4.1 (OpenAI) → AI Response             │
└───────────────────────────────────────────────┘
    │
    ▼
┌─ Persistence ────────────────────────────────┐
│  PostgreSQL + JSON Logs + Correlation IDs     │
└───────────────────────────────────────────────┘

Tech Stack

Table of Contents

1. Architecture
2. Tech Stack
3. What's New in v0.5.0
4. Installation
5. Configuration
6. Quick Start
7. Integration Examples
8. Decorators
9. REST API
10. SDK Client
11. ML Routers
12. Intent Detection
13. Feature Flags
14. PostgreSQL Logging
15. Logging & Observability
16. Streamlit Dashboard
17. Environment Variables
18. Troubleshooting
19. Development
20. Project Structure
21. License

What's New in v0.5.0

Multi-model routing

Ignis Router now supports smarter multi-model and multimodal routing across different query types and content formats. The router evaluates the request by intent, complexity, supported capabilities, and modality before choosing the best model.

Supported modalities include:

• TEXT
• IMAGE
• AUDIO
• VIDEO
• DOCUMENT

This makes it easier to route mixed-content workloads to the most suitable model without hard-coded one-size-fits-all behavior.

Expanded ML router support

This release expands the routing library with additional ML-based model selectors, including:

• elo
• hybrid_llm
• knn_multiround
• mlp

These routers improve model prediction quality and give teams more flexibility depending on cost, latency, and quality goals.

Security and release hardening

The 0.5.0 release also includes repository security cleanup and release hygiene improvements.

Observability and control

• richer routing decision metadata
• better budget guardrails for cost control
• improved API and dashboard visibility for routing performance and spend

Key Features

Installation

pip install git+https://github.com/Infogain-GenAI/ignis_router.git@main

pip install "ignis_router[all]"         # All LLM providers (OpenAI + Anthropic + Gemini)
pip install "ignis_router[dashboard]"   # Streamlit dashboard
pip install "ignis_router[dev]"         # Development tools (pytest, black, ruff)

For development:

git clone https://github.com/Infogain-GenAI/ignis_router.git
cd ignis_router
pip install -e ".[dev,all,dashboard]"

Requirements: Python ≥ 3.10, at least one LLM API key. PostgreSQL ≥ 14 is optional (for DB logging).

Configuration

Create a .env file in your project root:

# LLM Provider API Keys (set at least one)
OPENAI_API_KEY=sk-your-openai-key-here
# ANTHROPIC_API_KEY=sk-ant-your-key-here
# GOOGLE_API_KEY=your-google-api-key-here

# Routing strategy: quality-first, cost-first, latency-first, or balanced
ROUTER_YAML_CONFIG=configs/cost-first.yaml

# ML Router type: knn, svm, graph, or mf
ML_ROUTER_TYPE=svm
ENABLE_ML_MODEL_HINT_ROUTING=true
ML_CONFIDENCE_THRESHOLD=0.50

# Intent detection
ENABLE_ML_INTENT_DETECTION=true
ENABLE_RULE_BASED_INTENT_DETECTION=true

# PostgreSQL (optional — for persisting routing decisions)
ROUTER_DATABASE_URL=postgresql://postgres:your_password@localhost:5432/llm_router

# Logging
IGNIS_LOG_FILE=logs/ignis_router.log
IGNIS_LOG_CONSOLE=false

Routing strategies

Budget controls

Ignis Router includes a lightweight in-memory cost tracker that monitors spend across hourly and daily windows. When a budget is exceeded, the router can surface a warning or downgrade usage toward the cheapest available model.

# Optional cost budgets in USD
COST_BUDGET_DAILY_USD=50.00
COST_BUDGET_HOURLY_USD=5.00

The tracker uses model token costs to estimate spend and exposes live status via the API:

curl http://localhost:8080/budget

Sample response:

{
  "enabled": true,
  "daily": {
    "budget_usd": 50.0,
    "spent_usd": 12.34,
    "remaining_usd": 37.66,
    "percentage": 24.68,
    "state": "normal",
    "resets_at": "2026-09-11T00:00:00+00:00"
  },
  "hourly": {
    "budget_usd": 5.0,
    "spent_usd": 1.25,
    "remaining_usd": 3.75,
    "percentage": 25.0,
    "state": "normal",
    "resets_at": "2026-09-10T13:00:00+00:00"
  },
  "state": "normal",
  "action": "Normal operation"
}

When the budget is exceeded, the API returns 429 for /chat requests and the BudgetTracker marks the window as exceeded.

Quick Start

1. Create .env

OPENAI_API_KEY=YOUR_OPENAI_API_KEY_HERE
ENABLE_ML_INTENT_DETECTION=true
ENABLE_RULE_BASED_INTENT_DETECTION=true
ENABLE_ML_MODEL_HINT_ROUTING=true
ML_CONFIDENCE_THRESHOLD=0.50
ROUTER_YAML_CONFIG=configs/latency-first.yaml
ML_ROUTER_TYPE=svm

2. Route + Call LLM (Decorator)

from ignis_router import chat

@chat(system_prompt="You are a helpful assistant")
def ask(query, response):
    rd = response["routing_decision"]
    print(f"ML Predicted:  {rd['ml_router_predicted']}")
    print(f"Final Model:   {rd['final_model']}")
    print(f"Intent:        {rd['intent']}")
    print(f"Response:      {response['content'][:100]}")
    return response

ask("Write a Python function to sort a list")

3. Output

ML Predicted:  qwen2.5-7b-instruct
Final Model:   gpt-4.1-2025-04-14 (openai)
Intent:        code_generation
Response:      Here's a Python sorting function...

4. Route Only (No LLM call)

from ignis_router import Router

router = Router()
router.register_supported_models()
router.register_default_intent_rules()

result = router.route("Write a Python function to sort a list")
print(result.selected_model.model_name)   # claude-3-5-sonnet
print(result.detected_intent.value)       # code_generation
print(result.confidence)                  # 0.85

5. Route + Call LLM (Direct API)

from ignis_router import Router

router = Router()
router.register_supported_models()
router.register_default_intent_rules()
router.enable_llm_clients()

response = router.chat("Write a Python function to sort a list")
print(response["content"])     # AI response text
print(response["model"])       # gpt-4.1-2025-04-14
print(response["provider"])    # openai

Integration Examples

Example 1: Standalone Script

"""Use ignis_router in any Python script."""
from dotenv import load_dotenv
from ignis_router import chat

load_dotenv()


@chat(system_prompt="You are a senior Python developer")
def code_assistant(query, response):
    rd = response["routing_decision"]
    print(f"--- Routing Decision ---")
    print(f"ML Router Predicted:   {rd['ml_router_predicted']}")
    print(f"Rule-Based Would Pick: {rd['rule_based_would_pick']}")
    print(f"Final Model Used:      {rd['final_model']}")
    if rd["note"]:
        print(f"Note:                  {rd['note']}")
    print(f"Intent:                {rd['intent']}")
    print(f"Confidence:            {rd['confidence']:.2f}")
    print(f"Tokens:                {rd['tokens']}")
    print(f"\n--- Response ---")
    print(response["content"])
    return response["content"]


# Each call automatically routes to the best model
code_assistant("Write a REST API with authentication in FastAPI")
code_assistant("What is the time complexity of quicksort?")
code_assistant("Translate 'hello world' to French")

Output:

--- Routing Decision ---
ML Router Predicted:   qwen2.5-7b-instruct
Rule-Based Would Pick: claude-3-5-sonnet (intent rule: code_generation)
Final Model Used:      gpt-4.1-2025-04-14 (openai)
Note:                  API key not available for 'qwen2.5-7b-instruct', switched to available provider.
Intent:                code_generation
Confidence:            0.80
Tokens:                670

--- Response ---
Here's a FastAPI REST API with JWT authentication...

Example 2: Interactive AI Chat

"""Interactive terminal chat with routing visibility."""
from dotenv import load_dotenv
from ignis_router import Router

load_dotenv()

router = Router()
router.register_supported_models()
router.register_default_intent_rules()
router.enable_llm_clients()

print("Type your query (or 'exit' to quit):\n")

while True:
    query = input("You: ").strip()
    if query.lower() in ("exit", "quit"):
        break

    response = router.chat(query)
    routing = response.get("routing", {})

    print(f"\n  [Model: {response['model']} | Intent: {routing.get('intent')} | "
          f"Confidence: {routing.get('confidence', 0):.2f}]")
    print(f"\nAssistant: {response['content']}\n")

Example 3: FastAPI App

"""Add intelligent LLM routing to your existing FastAPI app."""
from fastapi import FastAPI
from dotenv import load_dotenv
from ignis_router import Router

load_dotenv()
app = FastAPI()

# Initialize router once at startup
router = Router()
router.register_supported_models()
router.register_default_intent_rules()
router.enable_llm_clients()


@app.post("/ask")
async def ask(query: str):
    response = router.chat(query)
    return {
        "answer": response["content"],
        "model_used": response["model"],
        "provider": response["provider"],
        "intent": response["routing"]["intent"],
        "confidence": response["routing"]["confidence"],
    }


@app.post("/route-only")
async def route_only(query: str):
    result = router.route(query)
    return {
        "best_model": result.selected_model.model_name,
        "intent": result.detected_intent.value,
        "confidence": result.confidence,
    }

Example 4: Flask App

"""Add intelligent LLM routing to your Flask app."""
from flask import Flask, request, jsonify
from dotenv import load_dotenv
from ignis_router import Router

load_dotenv()
app = Flask(__name__)

router = Router()
router.register_supported_models()
router.register_default_intent_rules()
router.enable_llm_clients()


@app.route("/chat", methods=["POST"])
def chat():
    query = request.json["query"]
    response = router.chat(query)
    return jsonify({
        "answer": response["content"],
        "model": response["model"],
        "routing": response.get("routing", {}),
    })


if __name__ == "__main__":
    app.run(port=5000)

Decorators

@route() — Route only (no LLM call)

from ignis_router import route

@route()
def handle(query, routing_result, routing_decision):
    print(f"Model: {routing_decision['final_model']}")
    print(f"Intent: {routing_decision['intent']}")
    return routing_decision

handle("Write Python code")

@chat() — Route + call LLM

from ignis_router import chat

@chat(system_prompt="You are a coding expert")
def ask(query, response):
    print(f"Model: {response['routing_decision']['final_model']}")
    print(f"Response: {response['content'][:200]}")
    return response["content"]

ask("Write code for API creation")

@with_router() — Inject configured router

from ignis_router import with_router

@with_router(enable_llm=True)
def my_app(router):
    result = router.chat("Explain quantum computing")
    print(result["content"])

my_app()

@retry() — Automatic retry

from ignis_router import retry, chat

@retry(max_attempts=3)
@chat()
def safe_ask(query, response):
    return response["content"]

Routing decision fields

REST API

Start the server

python -m ignis_router.api.run_api

Starts at http://127.0.0.1:8080. Swagger UI at /docs.

# Custom port
$env:API_PORT=9000; python -m ignis_router.api.run_api

# With DB connection string
$env:ROUTER_DATABASE_URL = 'postgresql://postgres:your_password@localhost:5432/llm_router'; python -m ignis_router.api.run_api

Endpoints

POST /route

// Request
{"query": "Write Python code for sorting"}

// Response
{"selected_model": "claude-3-5-sonnet", "strategy": "cost-first", "confidence": 0.8}

POST /chat

// Request
{"query": "Write Python code for sorting", "max_tokens": 1024, "temperature": 0.7}

// Response
{
  "content": "Here's a Python sorting function...",
  "model": "gpt-4.1-2025-04-14",
  "provider": "openai",
  "usage": {"prompt_tokens": 15, "completion_tokens": 120, "total_tokens": 135},
  "routing_decision": {
    "ml_router_predicted": "qwen2.5-7b-instruct",
    "rule_based_would_pick": "claude-3-5-sonnet",
    "final_model": "gpt-4.1-2025-04-14 (openai)",
    "intent": "code_generation",
    "confidence": 0.8,
    "tokens": 135
  }
}

SDK Client

from ignis_router import IgnisClient

with IgnisClient("http://127.0.0.1:8080") as client:
    # Route only
    result = client.route("Write Python code")
    print(result.selected_model)

    # Route + execute LLM
    chat = client.chat("Write Python code", max_tokens=512)
    print(chat.content)
    print(chat.model)

ML Routers

Four pre-trained routers from LLMRouter (open-source, UIUC):

ML_ROUTER_TYPE=svm

Which router for which use case?

Recommended .env by environment

Development / Testing:

ML_ROUTER_TYPE=knn
ENABLE_ML_MODEL_HINT_ROUTING=true
ML_CONFIDENCE_THRESHOLD=0.50
ROUTER_YAML_CONFIG=configs/balanced.yaml

Production (SaaS):

ML_ROUTER_TYPE=svm
ENABLE_ML_MODEL_HINT_ROUTING=true
ML_CONFIDENCE_THRESHOLD=0.60
ROUTER_YAML_CONFIG=configs/cost-first.yaml

Enterprise:

ML_ROUTER_TYPE=graph
ENABLE_ML_MODEL_HINT_ROUTING=true
ML_CONFIDENCE_THRESHOLD=0.70
ROUTER_YAML_CONFIG=configs/quality-first.yaml

When the predicted model's API key is unavailable, Ignis Router automatically falls back to an available provider.

Retraining

python -m ignis_router.scripts.train_all_routers        # All routers
python -m ignis_router.scripts.train_all_routers svm    # Specific router

Or programmatically:

from ignis_router import TrainingPipeline

pipeline = TrainingPipeline()
pipeline.train("svm")      # Train one
pipeline.train_all()       # Train all

Intent Detection

Ignis Router uses a hybrid intent detection system with two layers:

Layer 1: Semantic ML Classifier (primary)

• Uses Sentence Transformer embeddings + Logistic Regression
• Trained on data/intent_training_data.json
• If confidence ≥ ML_CONFIDENCE_THRESHOLD → uses ML result
• If confidence < threshold → falls back to Layer 2

Layer 2: Rule-Based Detector (fallback)

• Regex keyword matching (instant, < 1 ms)
• Always available, no model loading required

Supported intents and their default models

Configuration

# Both enabled = Hybrid mode (recommended)
ENABLE_ML_INTENT_DETECTION=true
ENABLE_RULE_BASED_INTENT_DETECTION=true

# Lower this if ML is too uncertain for your queries
ML_CONFIDENCE_THRESHOLD=0.50

What happens in practice

Query: "Write a REST API with authentication"

1. Semantic ML Classifier → confidence = 0.92 → intent = code_generation ✓
2. ML Router (SVM) → predicts: qwen2.5-7b-instruct
3. API key check → qwen2.5 not available → fallback to gpt-4.1 (OpenAI)
4. Strategy scoring → gpt-4.1 wins with quality-first weights
5. Call OpenAI → return response

Query: "hello how are you"

1. Semantic ML Classifier → confidence = 0.21 → too low!
2. Falls back to Rule-Based → intent = general_chat
3. ML Router → predicts: gemma-2-9b-it
4. API key check → not available → fallback to gpt-4o-mini (OpenAI)
5. Strategy scoring → gpt-4o-mini (cheapest for simple queries)
6. Call OpenAI → return response

Feature Flags

Toggle routing behavior at runtime without restarting the server or changing code.

Available flags

Usage

# Disable ML (use only rule-based)
curl -X PUT "http://localhost:8080/features/ml_based_routing?enabled=false"

# View current flags
curl http://localhost:8080/features

from ignis_router import Router, FeatureFlags

router = Router()
flags = FeatureFlags.from_config(router.config)
flags.set("enable_ml_model_hint_routing", False)
print(flags.to_dict())

PostgreSQL Logging

Every routing decision is automatically persisted to PostgreSQL — both via decorators and the REST API.

Setup

1. Create the database:

CREATE DATABASE llm_router;

2. Configure in .env:

ROUTER_DATABASE_URL=postgresql://postgres:your_password@localhost:5432/llm_router

The table is created automatically on first use. No manual schema needed.

What gets stored

Querying the data

-- Recent routing decisions
SELECT query_text, default_model_used, intent, confidence
FROM routing_responses
ORDER BY created_at DESC
LIMIT 10;

-- Model usage distribution
SELECT default_model_used, COUNT(*) as queries
FROM routing_responses
GROUP BY default_model_used
ORDER BY queries DESC;

-- Average confidence by intent
SELECT intent, AVG(confidence) as avg_confidence, COUNT(*) as count
FROM routing_responses
GROUP BY intent;

Logging & Observability

Structured JSON logs with correlation IDs.

IGNIS_LOG_FILE=logs/ignis_router.log
IGNIS_LOG_CONSOLE=false
IGNIS_LOG_LEVEL=INFO
IGNIS_LOG_FORMAT=json

Example log entry:

{
  "timestamp": "2026-07-24T05:40:48+00:00",
  "level": "INFO",
  "correlation_id": "cfd5595db1b749e5",
  "event": "routing_decision",
  "selected_model": "gpt-4.1-2025-04-14",
  "intent": "code_generation",
  "confidence": 0.85,
  "latency_ms": 15.3
}

Errors include full tracebacks with file, line, and function. Compatible with ELK, Datadog, CloudWatch, and Splunk.

from ignis_router import correlation_context

with correlation_context("my-trace-id") as cid:
    result = router.route("Write code")
    # All logs share the same correlation_id

Streamlit Dashboard

pip install "ignis_router[dashboard]"
python -m streamlit run examples/streamlit_dashboard.py

Opens at http://localhost:8501. Shows KPIs, model distribution, confidence charts, per-model/per-intent performance, direct vs routed latency, ML vs rule-based comparison, and routing log.

Requires both the API and PostgreSQL to be running.

Routing Latency Benchmark

Set at least one provider API key and ROUTER_DATABASE_URL, then run the reproducible benchmark:

ignis-benchmark-latency

The default run executes 50 direct and 50 routed calls across code, creative, reasoning, chat, and summarization intents. Each call is stored in benchmark_results; text and JSON reports are written to benchmark_reports/.

Choose the direct baseline model or skip PostgreSQL when needed:

ignis-benchmark-latency --provider openai --model gpt-4o-mini
ignis-benchmark-latency --no-db
py examples/direct_vs_routed.py

The direct path calls the provider client without Router.route(). The routed path uses the complete Router.chat() flow and reports routing, LLM, and total latency separately.

Environment Variables

Troubleshooting

Development

git clone https://github.com/Infogain-GenAI/ignis_router.git
cd ignis_router
pip install -e ".[dev,all,dashboard]"
python -m pytest tests/ -v

Project Structure

src/ignis_router/
├── api/           # FastAPI REST service + SDK client
├── configs/       # Routing strategy YAMLs + ML router configs
├── core/          # Router, routing engine, model selector
├── data/          # Intent training data
├── db/            # PostgreSQL persistence
├── detection/     # Intent detection (semantic + rule-based)
├── evaluation/    # Metrics, dashboard, reports
├── llm/           # LLM provider clients (OpenAI, Anthropic, Gemini)
├── ml/            # LLMRouter integration + ML inference
├── models/        # Pre-trained ML router models (.pkl, .pt)
└── scripts/       # Training scripts

Built by Infogain GenAI