DriftGuard

Observability for LLM applications and AI agents: catch drift, find the tool that's blocking your agent, and count the tokens it's wasting.

When an agent keeps retrying a failing tool, you pay for every attempt and the task never finishes. DriftGuard receives one event per tool call and names the blocking tool. It totals the wasted tokens and alerts you, for example: "The terminal tool failed 3 times in a row on task fix-tests; 6,600 tokens were spent on failed or redundant attempts." It also scores ordinary LLM traffic for drift: prompt and context bloat, falling retrieval relevance, and falling response quality.

• Agent diagnosis. Per-task status (blocked, failing, recovered, healthy), the blocking tool, repeated and redundant attempts, wasted tokens, and advice based on the error.
• Activity logs. A searchable, live history of every tool call, model call, prompt and session, with CSV/JSON export. Inputs and outputs are stored only if you opt in per project.
• Drift scoring. Four normalized metrics are checked against per-project thresholds and turned into a severity, a root cause, and a recommended mitigation. The Overview also compares each window with the period before it, and says so when there are too few events to judge.
• Alerts that manage themselves. Critical drift and blocked tasks raise alerts. Agent alerts resolve themselves when the task recovers. Alerts are pushed live to the dashboard, Slack, or any webhook.
• Any agent, no code. An driftguard-mcp-proxy gateway records any MCP client's tool calls from the middle of the stdio stream.
• Framework adapters. LangChain/LangGraph callbacks, the OpenAI Agents SDK, and the Claude Agent SDK, each one or two lines to wire in.
• Drop-in adapters. A @driftguard_tool decorator, MCP session instrumentation, token extraction from OpenAI, Anthropic, and Gemini responses, and a no-code Claude Code hook.
• Fast to operate. A status plate and a track diagram tell you what is blocked first. Keyboard shortcuts, links that carry your filters, bulk resolve, and a Policy page that shows what each threshold would flag.
• Self-hostable. A FastAPI server, a React dashboard, and SQLite or PostgreSQL, with Redis when you scale out. One docker compose up.

DriftGuard observes what your code reports, and it recommends actions rather than taking them. It doesn't intercept closed agents such as GitHub Copilot, and it never changes prompts or providers. See what it can't observe.

Start here

Every view is described in the dashboard guide.

Quickstart (SDK)

pip install "driftguard-tokepi[server]"   # PyPI name is driftguard-tokepi; you still import driftguard
driftguard-server                    # dashboard and API at http://127.0.0.1:8000

Create an account and a project in the dashboard, and copy the project API key. Then, in your app:

from driftguard import DriftGuardClient
from driftguard.adapters import AgentContext, driftguard_tool

client = DriftGuardClient(api_key="dg_live_...", project_id="coding-agent",
                          base_url="http://127.0.0.1:8000")

# LLM telemetry
client.capture_metrics(prompt_tokens=5200, context_length=6100,
                       retrieval_score=0.31, response_quality=0.62)
client.check_drift()   # {'severity': 'critical', 'risk_score': 1.0, 'actions': [...], ...}
client.sync_metrics()

# Agent tool calls
@driftguard_tool("terminal")
def run(cmd): ...

with AgentContext(client, task_id="fix-tests", auto_sync=True) as ctx:
    ctx.record_llm_usage(llm_response)   # attach model tokens to the next tool call
    run("pytest -x")

client.fetch_agent_diagnosis()["diagnosis"]

The SDK itself (pip install driftguard-tokepi) depends only on httpx. The full walkthrough, including Docker, is in docs/quickstart.md.

Documentation

Repository layout

driftguard/     the Python package: SDK, adapters, and driftguard.server
frontend/       dashboard source (React + Vite); builds into driftguard/server/static
research/       the baseline study (simulation) and the Phase 1 drift prototypes
integrations/   no-code integrations, starting with Claude Code hooks
tests/          SDK and server test suite
examples/       runnable scripts, including the live demo (demo.bat runs it on Windows)
docs/           documentation, including PRODUCT.md and the screenshots in docs/images
DESIGN.md       the dashboard's design tokens and components
.impeccable/    design direction and critique records

Development

pip install -e ".[dev]"
pytest                                   # SQLite; see docs/development.md for Postgres/Redis
ruff check . && ruff format --check .
cd frontend && npm install && npm run dev

License

MIT. See LICENSE.