ZidoRouter

Self-hosted gateway for LLM APIs. One process, zero dependencies beyond Python stdlib.

Point any OpenAI-compatible client at it and get: provider failover, circuit breakers, prompt caching, token savings, per-key metering, budget caps, and a live dashboard.

Install

Requires Python 3.9+.

pip install .

Quickstart

1. Add a provider (repeat for each upstream you have):

zidorouter add groq https://api.groq.com/openai/v1 gpt-4o-mini YOUR_GROQ_KEY

2. Set a dashboard password:

zidorouter dashboard-pass

3. Serve:

zidorouter serve 8764

Now point your client at http://127.0.0.1:8764/v1 and use any dashboard-issued key (zid-...) as the API key.

Dashboard

Open http://127.0.0.1:8764/ and log in with the password you set. From there:

• add or remove providers, toggle them on or off
• issue and revoke client keys
• set model aliases (combos) and per-provider prices
• watch usage, cache hit rate, and audit logs live

Features

• Multi-provider pool with automatic failover and per-provider circuit breakers
• Health probes and status at a glance
• Auto prompt-cache markers (opt out per request or globally)
• Context slimming: trims old tool output and folds mid-history so tokens are never sent twice
• Per-key rate limits, daily budgets, and usage metering
• Cost estimation per provider
• Webhooks for failover, budget exceeded, and provider recovery events
• Audit log of every request

CLI

zidorouter list                 # show providers
zidorouter add NAME BASE MODEL KEY [NOTES]
zidorouter remove NAME
zidorouter dashboard-pass       # set or change the dashboard password
zidorouter serve [PORT] [HOST]  # default: 8764 on 127.0.0.1

Running as a service (systemd)

[Unit]
Description=ZidoRouter
After=network.target

[Service]
ExecStart=/usr/bin/env zidorouter serve 8764
Restart=always

[Install]
WantedBy=multi-user.target

Notes

State lives in ~/.zidorouter/ (keys, aliases, audit log, metrics). Back that folder up and you can move the whole gateway to a new machine in minutes.