llm-leash 2.20.0

Cost ceiling, audit log, and kill switch for LLM agents.

llm-leash

The cost ceiling, audit log, and kill switch your LLM agent should never run without.

llm-leash is a 5-line runtime firewall for LLM agents. It enforces hard USD budgets, writes an immutable audit log, and gives you a kill switch and human-in-the-loop hook — without locking you into any agent framework.

Status: v2.20.0 — Production / Stable. Both the in-process middleware (v1.x surface, unchanged) and the HTTP proxy + operator console (v2.x surface) are part of the semver-stable public API. See CHANGELOG.md.

Why

You shipped an agent. Then:

• A retry loop spent $2,387 in 14 minutes.
• A vague user message coaxed it into DROP TABLE users.
• Compliance asked "show me every action this agent took for customer X last month" — you can't.

llm-leash solves the boring B2B half of agent safety that nobody else owns: money, paperwork, panic button. It works alongside the content-safety scanners (LlamaFirewall, Invariant, Prompt-Guard) — not against them.

Two ways to install the leash

Mode	When	Code change required
In-process middleware (v1.x surface, stable)	Python agents you control the source of	1 line: fw.wrap(client)
HTTP proxy (v2.0 GA, stable)	Any language, vendor agents, multi-app compliance	0 lines — one env var: OPENAI_BASE_URL=http://localhost:8000

Both modes share the same audit format, policy engine, secrets detection, HITL queue, semantic detection (LLMGuardRule), behavioral baseline (BehavioralBaselineRule), and Prometheus metrics. Pick by code-access constraints, not by features.

Quickstart

from llm_leash import Firewall, LeashKilled
from anthropic import Anthropic

fw = Firewall(budget_usd=10.00, audit_log="audit.jsonl")
client = fw.wrap(Anthropic())

try:
    while True:
        client.messages.create(model="claude-opus-4-7", max_tokens=200,
                               messages=[{"role": "user", "content": "..."}])
except LeashKilled as e:
    print(f"Saved you the rest. Reason: {e.reason}")

Three things happen on every call:

1. Budget tracked — cumulative cost per session, raises LeashKilled when the cap is hit.
2. Audit logged — every model call appends one hash-chained JSONL line. Tamper-evident: llm-leash verify audit.jsonl.
3. Kill switch — await fw.kill("reason") stops the session immediately; the next call raises LeashKilled.

Run the offline demo (no API key needed):

python demo.py
llm-leash verify audit.jsonl

Proxy mode (v2.0 GA)

For agents you can't (or don't want to) modify — change one env var, get the firewall:

# Install + start
pip install "llm-leash[proxy]"
llm-leash-proxy --listen 127.0.0.1:8000 --audit-log audit.jsonl --budget-usd 50

# Point any agent at it (no source changes)
export ANTHROPIC_BASE_URL=http://localhost:8000
export OPENAI_BASE_URL=http://localhost:8000
python my_agent.py

Works with any client speaking OpenAI / Anthropic on-wire protocol: the OpenAI / Anthropic SDKs, OpenRouter, LangChain.js, Vercel AI SDK, CrewAI via LiteLLM, OpenHands, custom Rust / Go / TS agents. Identifies sessions via headers (X-LLM-Leash-Session-Id, X-LLM-Leash-Tenant-Id, X-LLM-Leash-Agent-Name), with auto-fallback to a hash of the bearer token when no headers are set.

What v2.0 GA gives you over v1.x:

• Streaming SSE with end-of-stream accounting and mid-stream cancellation — if a runaway generation would blow the hard cap mid-flight, the proxy aborts the upstream connection and emits a synthetic event: error frame to the client.
• Multi-replica state sharing via Redis backend (budget.backend = "redis", kill.backend = "redis") — run the proxy behind a load balancer with consistent budget cumulatives + kill state across pods.
• Per-agent budget caps with [budget.per_agent_caps] TOML — agent-level ceilings override per-tenant, which override the default cap.
• Operator console — llm-leash-console ships a read-only Web UI with live sessions, top spend, kill button, human-review queue, trends charts, audit tail. See the Operator console section below.
• Alerts sidecar — llm-leash-alerts watches the audit stream and fans budget-breach / kill / HITL events out to Slack + PagerDuty.
• Docker + Helm + k8s manifests for fleet deployment.

Operator workflow:

# Real-time stats from the CLI
llm-leash status --proxy http://localhost:8000

# Stop a runaway session
llm-leash kill <session_id> --proxy http://localhost:8000 --reason "blew budget"

# Approve a pending HITL request
llm-leash hitl list --proxy http://localhost:8000
llm-leash hitl approve <request_id> --proxy http://localhost:8000

# Prometheus scrape
curl http://localhost:8000/metrics

# Operator Web UI (separate binary, separate port so a UI crash never
# takes down agent traffic)
llm-leash-console --proxy http://localhost:8000 --listen 127.0.0.1:8001

See docs/PROXY.md for deployment recipes (Docker, k8s, Helm), Redis tuning, and the full TOML reference.

Operator console (v2.14 → v2.18)

A read-only Web UI (llm-leash-console) that visualises the proxy's live state and audit stream. Sits on its own port so a UI crash never takes down agent traffic.

Features at a glance:

• Sticky navigation bar with live counters for human-review queue, active sessions and threats — red urgency marker when there's something to look at.
• KPI strips — Threats prevented (HIGH / MEDIUM / LOW / review queue) and Proxy live state (active sessions / spend / rules / PII redactor).
• Trends panel — inline SVG charts: spend per hour over the last 24 h, threats by agent for the selected period (click a bar → opens the agent drawer).
• Human review queue — paused requests waiting for approval; approve or reject one at a time or in bulk.
• Active sessions — top-spend sessions with inline kill button (with confirmation).
• Threats by rule + Threat detail — every policy decision, sorted by risk; click any row for full context.
• Per-agent spend + Per-agent budget caps editor.
• Export — one-click CSV (threats) and JSON (audit) downloads, ready for SOC 2 evidence binders.
• Recent events — live WebSocket tail of the audit log.

Detail drawer (v2.17)

Click any row in the threat / session / human-review / agent tables to open a 480 px side panel with the full event JSON, related events from the same session/agent, and inline contextual actions (approve, reject, kill, copy deep link). Keyboard navigation: Esc closes, ↑ / ↓ cycle through rows — same UX pattern as Linear, Datadog Logs, Sentry, Langsmith.

Bulk actions, filters, dark mode (v2.18)

Checkbox column on Human review queue and Active sessions tables → bulk approve / reject / kill. Free-text search above every table. Manual dark-mode toggle (auto / dark / light, persisted to localStorage).

Charts panel — spend & threats (v2.18)

Running it

pip install "llm-leash[proxy]"
llm-leash-proxy --listen 127.0.0.1:8000 --audit-log audit.jsonl
llm-leash-console --proxy http://localhost:8000 \
                  --admin-token "$LLM_LEASH_ADMIN_TOKEN" \
                  --audit-log audit.jsonl --port 8801

For production deployment (systemd, Docker, gunicorn multi-worker, nginx WS timeouts, k8s liveness probes), see docs/deployment.md.

Adapters — one wrap, every framework (in-process, stable)

from llm_leash import Firewall
fw = Firewall(budget_usd=10.00, audit_log="audit.jsonl")

Framework	Client class	Example
Anthropic	anthropic.Anthropic	fw.wrap(Anthropic()).messages.create(...)
OpenAI	openai.OpenAI	fw.wrap(OpenAI()).chat.completions.create(...)
LangChain / LangGraph	ChatAnthropic, ChatOpenAI	fw.wrap(ChatAnthropic(model="…")).invoke([…])
CrewAI	crewai.LLM	fw.wrap(LLM(model="openai/gpt-4o")).call([…])
OpenHands	openhands.llm.LLM	fw.wrap(LLM(config)).completion(messages=[…])
Pydantic-AI	pydantic_ai.models.*	await fw.wrap(OpenAIModel(...)).request([…])
MCP	mcp.ClientSession	await fw.wrap(session).call_tool("read_file", {…})

All adapters are duck-typed — no SDK imports at module level, no version pinning. The wrapped client preserves every attribute of the original; only the call surface is intercepted.

# LangGraph example: drop the firewall into your existing graph
from langchain_anthropic import ChatAnthropic
from langgraph.graph import StateGraph

chat = fw.wrap(ChatAnthropic(model="claude-haiku-4-5"))
graph = StateGraph(MyState)
graph.add_node("llm", lambda state: {"reply": chat.invoke(state["messages"])})

# CrewAI example: pass the wrapped LLM to your Agent
from crewai import Agent, Crew, Task, LLM
llm = fw.wrap(LLM(model="anthropic/claude-haiku-4-5"))
agent = Agent(role="researcher", llm=llm, goal="...")
result = Crew(agents=[agent], tasks=[Task(...)]).kickoff()

# MCP example: every tool call is audited; dangerous tools can require HITL
async with ClientSession(read, write) as session:
    wrapped = fw.wrap(session)
    await wrapped.call_tool("read_file", {"path": "/etc/hosts"})

Pre-push leak prevention

Block accidental commits of internal-tooling state (.great_cto/, .claude/, .beads/), repo-boundary paths (/Users/<name>/...), and git-managed secret files (.env, id_rsa, .aws/credentials) before they reach a public remote. Same rule, two surfaces:

from llm_leash import ArtifactLeakageRule, Firewall

fw = Firewall(rules=[ArtifactLeakageRule(action="block")])
# block | hitl | redact

# As a git pre-push hook:
cp examples/pre-push-hook.sh .git/hooks/pre-push && chmod +x .git/hooks/pre-push

# Or manually:
llm-leash scan --staged                  # current git diff --cached
llm-leash scan --push-range A..B         # commits about to be pushed
llm-leash scan src/ tests/               # arbitrary paths

See docs/LEAKAGE.md for the full detector list, CI recipes, and waiver syntax.

SOC 2 evidence pack

Generate a complete SOC 2 evidence package from any audit.jsonl log:

llm-leash soc2 /var/log/agent-audit.jsonl \
  --out ./evidence-2026-Q2/ \
  --period-start 2026-04-01T00:00:00Z \
  --period-end   2026-06-30T23:59:59Z \
  --org "Acme Inc"

Produces six artefacts an auditor can attach to their evidence binder directly: executive-summary.html, cc6_access_control.csv, cc7_monitoring.csv, cc7_integrity.json, anomalies.csv, and bom.json. Each file is sha256-hashed and listed in the bill of materials. See docs/SOC2.md for the Trust Service Criteria mapping.

Persistent state for multi-worker prod

from llm_leash import Firewall, SQLiteBudgetStore, SQLiteKillRegistry

fw = Firewall(
    budget_usd=100.0,
    audit_log="/var/log/agent-audit.jsonl",
    kill_registry=SQLiteKillRegistry("/var/lib/myapp/kill.db"),
)
fw._budget._store = SQLiteBudgetStore("/var/lib/myapp/budget.db")

Redis variants (RedisBudgetStore / RedisKillRegistry) accept any duck-typed client.

For proxy mode, the same backends are config-driven — no code:

# proxy.toml
[budget]
backend = "redis"
redis_url = "redis://redis.internal:6379/0"
default_cap_usd = 50.0
per_tenant_caps = { acme = 500.0, beta = 25.0 }
per_agent_caps  = { "writer-prod" = 100.0, "researcher-prod" = 10.0 }

[kill]
backend = "redis"
redis_url = "redis://redis.internal:6379/0"

What it does

• Hard USD budget per session. Soft cap warns. Hard cap kills.
• Append-only JSONL audit log, hash-chained, optionally HMAC-signed. jq-able. SOC 2 / EU AI Act Article 12 evidence-shaped.
• Kill switch. Stop a runaway agent from CLI, HTTP, or Redis. Sub-300ms propagation.
• Human-in-the-loop webhook for high-stakes tool calls. Default-deny on timeout.
• Tool ACL with regex / SQL-AST / shell-AST patterns.
• PII redaction before tool dispatch and before audit write.
• Adapters for Anthropic, OpenAI, LangGraph, CrewAI, OpenHands, Pydantic-AI, MCP.
• Semantic threat detection (LLMGuardRule) — cheap-LLM classifier flags prompt injection / jailbreak / data exfiltration; sampling-rate aware so high-QPS is affordable.
• Indirect prompt-injection detection (ToolResultScanner) — scans tool-result content (web pages, file contents, DB rows) for embedded injection markers / role confusion / unicode obfuscation. Closes OWASP LLM01.
• Ensemble multi-rule scoring (EnsemblePolicyEngine) — weighted aggregation of all rule outputs; reduces false-positives by requiring corroboration across rules, while catching weak signals via summation.
• Behavioral baseline (BehavioralBaselineRule) — Welford-online statistics per (tenant, agent); flags token spikes, new models, off-hours, tool-spam. Pluggable InMemoryBaselineStore / SQLiteBaselineStore.
• Streaming + mid-stream cancel (proxy) for both Anthropic and OpenAI SSE.

What it does NOT do

You want	Use this instead
Prompt-injection classifier	Prompt-Guard (call from a rule)
Content guardrails (DSL)	NeMo Guardrails / Guardrails AI
Tool-arg pattern catalog	Invariant Labs (import their .rules from a policy)
Eval framework	PromptFoo / DeepEval
Observability dashboard	Langfuse / LangSmith (ship JSONL into them)
Model router	LiteLLM / OpenRouter

llm-leash is the layer beneath all of them. It does enforcement and evidence. Everything else is a rule you can plug in.

Documents

• PRODUCT.md — what this is, who buys it, what it is not.
• ARCHITECTURE.md — modules, data flow, performance budget.
• API.md — public surface, CLI, JSONL schema, custom rules.
• docs/PROXY.md — proxy mode operator guide (Docker, k8s, Redis, TOML).
• docs/deployment.md — production deployment (systemd, Docker, gunicorn, nginx, k8s probes, synthetic monitoring).
• docs/SOC2.md — SOC 2 Trust Service Criteria mapping.
• docs/adr/ — architecture decisions (in progress).

Install

pip install llm-leash                  # core, zero runtime deps
pip install "llm-leash[anthropic]"     # + Anthropic adapter
pip install "llm-leash[proxy]"         # + HTTP proxy mode (starlette/uvicorn/httpx)
pip install "llm-leash[redis]"         # + Redis backend for proxy state
pip install "llm-leash[all]"           # everything

Roadmap

Version	Status	What
v0.1	✓	Core firewall + Anthropic adapter + audit chain + CLI verify
v0.2	✓	PolicyEngine + PII redactor
v0.3	✓	BlockedSql + BlockedShell rules
v0.4	✓	Redis transports for budget + kill
v0.5	✓	HITL gates (InMemory + Webhook) + HitlThreshold
v0.6	✓	LangGraph + CrewAI + MCP adapters + acceptance gate
v0.7	✓	audit replay/export + SQLite stores + extended CLI
v1.0	✓	Stable public API · semver lock · PyPI release · per-adapter examples
v1.1	✓	OpenHands + Pydantic-AI adapters · LlamaFirewall / Presidio rule wrappers
v1.2	✓	Durable HITL queue (SQLite/InMemory) · HTTP kill transport · CLI hitl ops
v1.3	✓	SOC 2 evidence pack generator · CLI soc2 · TSC mapping
v2.0	✓	HTTP proxy mode · SSE streaming + mid-stream cancel · Redis/SQLite backends · per-agent caps · operator console (llm-leash-console) · alerts sidecar (llm-leash-alerts) · LLMGuardRule (semantic) · BehavioralBaselineRule · Docker / k8s / Helm
v2.14	✓	Risk levels + human-readable explanations on every policy decision · /api/agents endpoint · "Threats prevented" KPI strip
v2.15	✓	Console: kill button per session · CSV/JSON export · spend sparkline · agent drill-down · human-review approve/reject panel
v2.16	✓	Console UX: timestamps with date · confirmation dialogs · sticky nav · payload viewer · section reorder · production resilience (uvicorn flags, gunicorn factory, deployment guide)
v2.17	✓	Detail drawer for event drill-down · keyboard navigation · URL deep links (?event=…)
v2.18	✓	Trends charts (spend over time + threats by agent) · bulk actions · table filters / search · manual dark-mode toggle
v2.19	✓	ToolResultScanner — indirect prompt-injection detection on tool-result content (OWASP LLM01)
v2.20	✓	EnsemblePolicyEngine — weighted multi-rule aggregation
v2.21	planned	OPA / Rego policy backend

License

MIT — see LICENSE.

The OSS firewall is and always will be free. The hosted audit-log service (forthcoming) is the only thing that costs money — and you never need it. JSONL is yours.