Skip to main content

DriftShield

Your LangChain agent just called the same API 47 times. Your CrewAI crew burned £200 in tokens overnight. Your research agent started writing marketing copy instead of financial summaries.

You didn't find out until morning.

DriftShield catches this stuff in real-time. It wraps your existing agent, watches what it does, and pings you on Slack or Discord the moment something goes sideways. No dashboard. No cloud. No account to create. Just a Python library that runs alongside your agent.


What it actually does

DriftShield monitors three things:

Loop detection: Is your agent calling the same tool over and over? Or stuck in a cycle like search → format → search → format? DriftShield spots the pattern and alerts you before it eats your budget.

Goal drift: Is your agent still doing what you asked it to? DriftShield uses local embeddings (runs on your CPU, no API calls) to measure how far the agent's output has drifted from its original objective.

Resource spikes: Is this run burning way more tokens or taking way longer than usual? DriftShield learns what "normal" looks like for your agent, then flags when things go abnormal.

Everything stays on your machine. Traces go to a local SQLite file. Embeddings run on your CPU. The only thing that leaves your machine is the alert you choose to send to Slack/Discord.


Get started

pip install driftshield-mini

LangChain

from driftshield import DriftMonitor

monitor = DriftMonitor(
    agent_id="logistics-v2",
    alert_webhook="https://hooks.slack.com/...",
)

agent = monitor.wrap(existing_agent)
result = agent.invoke({"input": "optimise route for order #4821"})
# DriftShield is now watching. That's it.

CrewAI

from driftshield.crewai import DriftCrew

crew = DriftCrew(
    crew=existing_crew,
    agent_id="research-team-v1",
    alert_webhook="https://discord.com/api/webhooks/...",
)

result = crew.kickoff()

Works with any LLM

OpenAI, Anthropic, Groq, Ollama, local models doesn't matter. DriftShield only sees the traces (tool calls, token counts, outputs), not the model internals. Swap providers whenever you want.


How calibration works

For the first 30 runs (configurable), DriftShield quietly observes your agent and builds a baseline average tokens per run, typical tool sequences, normal execution time. No alerts during this phase.

After that, it knows what "normal" looks like and starts flagging deviations. You can inspect the baseline anytime:

driftshield baseline my-agent

Tip: If 30 runs feels like a lot, you can lower calibration_runs or use a preset template. DriftShield still catches obvious problems (like 50 identical tool calls) even without a baseline, using absolute safety limits.


What an alert looks like

When drift hits your Slack/Discord, you get:

{
  "agent_id": "logistics-v2",
  "detector": "action_loop",
  "severity": "HIGH",
  "message": "Action loop: search_inventory called 6x in 45s",
  "suggested_action": "Check search_inventory input/output for stale data or error loops",
  "context": {
    "tool_name": "search_inventory",
    "repeat_count": 6,
    "recent_actions": ["search_inventory", "search_inventory", "search_inventory", "..."]
  }
}

Not just "something's wrong" — it tells you what happened, which detector caught it, and what to check first.


CLI

# What went wrong in the last 24 hours?
driftshield alerts --last 24h

# Show me exactly what my agent did on its last run
driftshield traces logistics-v2 --run latest

# What does "normal" look like for this agent?
driftshield baseline logistics-v2

# List recent runs
driftshield runs logistics-v2

Configuration

Everything's tuneable. Defaults are sensible, but you can adjust:

monitor = DriftMonitor(
    agent_id="my-agent",
    alert_webhook="https://hooks.slack.com/...",
    goal_description="Summarise financial reports",
    calibration_runs=30,         # runs before baseline kicks in
    loop_window=20,              # how many recent actions to check
    loop_max_repeats=4,          # repeated calls before flagging
    similarity_threshold=0.5,    # goal drift sensitivity (lower = stricter)
    spike_multiplier=2.5,        # how many std devs = a spike
    min_alert_severity="MED",    # ignore LOW severity events
    alert_cooldown=60.0,         # don't spam the same alert
)

Custom reactions

DriftShield alerts you by default, but you can also react programmatically:

def handle_drift(event):
    if event.severity.value == "CRITICAL":
        agent.stop()  # kill the run
        page_oncall()  # wake someone up

monitor.on_drift(handle_drift)

What this isn't

I want to be upfront about scope. DriftShield is v0.1.1, built by one person.

  • Not a full observability platform. No web dashboard, no hosted backend, no team features. If you need that, look at LangSmith, Langfuse, or Arize.
  • Not a guardrail system. It detects drift after the fact and alerts you. It doesn't block actions before they happen (that's on the roadmap).
  • Not production-hardened yet. It works, it's tested, but it hasn't been battle-tested by thousands of users. Expect rough edges.

What it IS: the smallest, simplest tool that does one thing well — tells you when your agent is going off the rails, fast, with zero setup overhead.


Roadmap

  • v0.2 — Auto-correction hooks (retry, context trim, kill run). Preset baseline templates so you get value from run 1.
  • v0.3 — Better multi-agent support. Predictive drift (catch it before it happens).
  • v1.0 — Dashboard, team features, historical analytics. But only if people actually want it.

Built with

  • Python 3.10+
  • SQLite (zero config)
  • sentence-transformers (local CPU embeddings)
  • scikit-learn (basic stats)
  • httpx (webhooks)
  • click + rich (CLI)

Contributing

This is early. If you're running agents in production and hit a case DriftShield missed (or flagged incorrectly), please open an issue. Your real-world edge cases are the most valuable thing you can give this project right now.

git clone https://github.com/YOUR_USERNAME/Driftshield-mini.git
cd driftshield
python -m venv .venv
source .venv/bin/activate  # or .venv\Scripts\activate on Windows
pip install -e ".[dev]"
python -m pytest tests/ -v

V2

Supported frameworks (v0.2.0)

All wrappers share the same detectors (action loop, goal drift, resource spike), local SQLite storage, and Slack/Discord alerting. Nothing leaves the machine.

from driftshield_mini import DriftMonitor                    # LangChain / manual API
from driftshield_mini.crewai import DriftCrew                 # CrewAI
from driftshield_mini.autogen import DriftAutogenAgent        # Microsoft AutoGen
from driftshield_mini.llama_index import DriftLlamaIndexHandler  # LlamaIndex
from driftshield_mini.openai_assistants import DriftOpenAIClient # OpenAI Assistants
from driftshield_mini.semantic_kernel import DriftKernelFilter   # Semantic Kernel
from driftshield_mini.haystack import DriftHaystackTracer        # Haystack
from driftshield_mini.google_adk import DriftADKCallbacks        # Google ADK

AutoGen (Microsoft)

monitored = DriftAutogenAgent(agent=assistant, agent_id="autogen-analyst-v1",
                              alert_webhook="https://hooks.slack.com/...")
user_proxy.initiate_chat(monitored.agent, message="Summarise Q3 results")

LlamaIndex

from llama_index.core.callbacks import CallbackManager
from llama_index.core import Settings
handler = DriftLlamaIndexHandler(agent_id="rag-agent-v1", goal_description="Answer policy questions")
Settings.callback_manager = CallbackManager([handler.callback_handler])

OpenAI Assistants

client = DriftOpenAIClient(openai.OpenAI(), agent_id="assistant-v1", goal_description="Process invoices")
client.run_assistant(thread_id=t.id, assistant_id=a.id)

Semantic Kernel

DriftKernelFilter(agent_id="sk-agent-v1", goal_description="Reconcile invoices").apply_to_kernel(kernel)

Haystack

DriftHaystackTracer(agent_id="haystack-v1").enable()   # once, before running pipelines

Google ADK

drift = DriftADKCallbacks(agent_id="adk-v1")
agent = Agent(..., before_tool_callback=drift.before_tool, after_tool_callback=drift.after_tool)

Compliance audit export

driftshield export --agent my-agent --output audit.csv
driftshield export --agent my-agent --output drift.json --drift-only --format json

Offline / air-gapped

driftshield download-model

Why I built this

I kept reading the same story: dev builds agent, agent works great in testing, agent goes haywire in production at 2am, dev wakes up to a hefty API bill and a Slack full of confused users. The big observability platforms exist but they're heavy on dashboards, accounts, pricing tiers, cloud dependencies. Most solo devs and small teams just want to know when their agent is broken. That's it.

So I built the smallest thing that solves that problem.

If you try it and it helps (or doesn't), I genuinely want to hear about it.


License

MIT - do whatever you want with it.

Metadata

Release files for driftshield-mini 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for driftshield-mini 0.2.0
File Size Uploaded Provenance Actions
driftshield_mini-0.2.0.tar.gz 37.0 kB Without provenance Details for driftshield_mini-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for driftshield-mini 0.2.0
File Interpreter ABI Platform Actions
driftshield_mini-0.2.0-py3-none-any.whl Python 3 none any Details for driftshield_mini-0.2.0-py3-none-any.whl

Total release size: 78.9 kB

Release files / driftshield_mini-0.2.0.tar.gz

Download URL driftshield_mini-0.2.0.tar.gz
Size 37.0 kB
Tags Source
SHA-256 checksum
How to use checksums
928f413fb3492006727cc1621a9992753882d6d19ef303fc5a8c005053f5f977
BLAKE2b-256 checksum
How to use checksums
a83d59142112e7d5834dadcf8634d8b1a49ac3c298823cb05dcb1b5f3613b438
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.9

Release files / driftshield_mini-0.2.0-py3-none-any.whl

Download URL driftshield_mini-0.2.0-py3-none-any.whl
Size 41.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d128740d5b705289ac34c28a625bdb8200963d78c3ee72076a26d96953348900
BLAKE2b-256 checksum
How to use checksums
b3d75af5e86458284d23dffe17b4d29d83121edce3a691cc45fec3b9be314920
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.9

Security for driftshield-mini 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page