localab-circuit-breaker 0.1.0

A tiny local proxy that hard-caps LLM API spend.

⚡ agent-circuit-breaker

A tiny, local, hard-cap spend guard for LLM agents.

"Set a budget, walk away, and never come back to a surprise bill."

---

🎬 What it does

agent-circuit-breaker is a local HTTP proxy that sits between your agent and any OpenAI- or Anthropic-compatible API. It enforces hard USD spend caps:

• Per-run budget — one agent session cannot spend more than you allow.
• Per-day budget — your total daily spend across all runs is capped.
• Global kill switch — instantly block all traffic with one command.

When a cap is hit, the proxy returns a clear 429 Budget exhausted response. It does not silently degrade, retry, or fall back to a cheaper model.

┌─────────────┐     ┌─────────────────────────────┐     ┌─────────────────┐
│   Agent     │────▶│  agent-circuit-breaker      │────▶│  Real provider  │
│  (SDK/CLI)  │     │  http://localhost:8989      │     │  OpenAI/Claude  │
└─────────────┘     └─────────────────────────────┘     └─────────────────┘
                              │
                              ▼
                    ┌─────────────────────┐
                    │  SQLite state file │
                    │  spend + kill flag │
                    └─────────────────────┘

---

🧠 Methodology: why this exists

Most cost-tracking tools (LiteLLM, etc.) are great at reporting spend. They are not designed to hard-stop a runaway agent mid-loop.

A runaway agent can:

• Get stuck in a retry loop.
• Spawn recursive tool calls.
• Call expensive models repeatedly while you are away.

Cost-tracking tools generally sit in one of three places:

1. They track spend after the fact.
2. They are too heavy to drop in front of a quick script.
3. They report, but do not refuse, the request.

agent-circuit-breaker aims at the narrow slot none of those cover: a single-purpose, lightweight guard that says no the moment the cap is reached.

---

🔍 How this compares

The agent-circuit-breaker name on PyPI belongs to a different project, and this one is not it:

• sagarchhatrala/agent-circuit-breaker — "a deterministic safety gate for AI coding agents", v1.4.8 across 15 releases. It gates what an agent is allowed to do. This tool gates how much it is allowed to spend. Adjacent problems, same name, different tool — theirs is the more mature project and it got there first.

Because that name is taken, this project installs as localab-circuit-breaker while the repository, module and agent-circuit-breaker CLI command keep their original names. If you want the safety gate, install theirs; if you want a hard USD cap, install this.

Honest scope: hard spend caps are not an unclaimed idea. LiteLLM and friends will happily tell you what you spent. What is genuinely uncommon is a proxy small enough to put in front of a throwaway script that returns 429 and stops, rather than logging the overspend and continuing.

Design principles

Principle	How we enforce it
Hard stop	Returns 429 immediately when a budget is exhausted.
Local-first	SQLite state file only. No cloud, no telemetry, no accounts.
Minimal surface	No web UI, no plugin system, no dashboards. Just a proxy.
User-controlled pricing	Bundled prices.json that you edit. No network price fetching.
Transparent	Passes through bytes unchanged; accounting happens on the response path.

---

🏗️ Architecture

flowchart LR
    A[Agent / SDK] -->|HTTP| B(agent-circuit-breaker)
    B -->|OpenAI format| C[OpenAI-compatible API]
    B -->|Anthropic format| D[Anthropic-compatible API]
    B --> E[(SQLite state)]
    
    style B fill:#f9f,stroke:#333,stroke-width:2px
    style E fill:#bbf,stroke:#333,stroke-width:2px

Request lifecycle

sequenceDiagram
    participant Agent
    participant ACB as agent-circuit-breaker
    participant SQLite
    participant Provider

    Agent->>ACB: POST /openai/v1/chat/completions
    ACB->>SQLite: Check kill switch
    ACB->>SQLite: Check run + daily budget
    alt Budget OK
        ACB->>Provider: Forward request
        Provider-->>ACB: Response + usage
        ACB->>SQLite: Record spend
        ACB-->>Agent: Return response
    else Budget exhausted
        ACB-->>Agent: 429 Budget exhausted
    end

---

📦 Installation

Recommended: pipx (global command)

# Windows
pipx install "<path-to-repo>"

# macOS / Linux
pipx install .

This makes agent-circuit-breaker available everywhere without activating a virtual environment.

Development: editable install

# Windows PowerShell
uv venv
uv pip install -e .
.venv\Scripts\agent-circuit-breaker --help

# macOS / Linux
uv venv
uv pip install -e .
.venv/bin/agent-circuit-breaker --help

Why the command was not found? If you installed with uv pip install -e . but did not activate the virtual environment, Windows does not know about .venv\Scripts. Either activate the venv with .venv\Scripts\Activate.ps1 or use the full path .venv\Scripts\agent-circuit-breaker.exe.

---

🚀 Quick start

1. Start the breaker

# Windows PowerShell
agent-circuit-breaker --run-budget 2.00 --daily-budget 20.00

You will see:

agent-circuit-breaker listening on http://127.0.0.1:8989 (run budget $2.00, daily $20.00)
Press Ctrl+C to stop.

2. Point your agent at it

# Windows PowerShell
$env:OPENAI_BASE_URL = "http://localhost:8989/openai"
$env:ANTHROPIC_BASE_URL = "http://localhost:8989/anthropic"

# macOS / Linux
export OPENAI_BASE_URL=http://localhost:8989/openai
export ANTHROPIC_BASE_URL=http://localhost:8989/anthropic

3. Run your agent as normal

The SDK will send requests through the proxy. The proxy forwards them to the real provider and counts the cost.

4. Test it with curl

curl http://localhost:8989/openai/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

---

🛑 Kill switch

Instantly block all traffic, regardless of budget:

agent-circuit-breaker --kill
# Kill switch ACTIVE

agent-circuit-breaker --unkill
# Kill switch INACTIVE

This is perfect for:

• Stopping a runaway agent without finding its process.
• Emergency shutdown when you notice unexpected spend.
• CI/CD pipelines where you want to guarantee no API calls.

---

⚙️ Configuration

CLI options

agent-circuit-breaker [options]

  --run-budget USD         Budget for this process (default: 2.00)
  --daily-budget USD       Budget per calendar day (default: 20.00)
  --state PATH             SQLite state file
                           (default: ~/.agent-circuit-breaker/state.db)
  --host HOST              Bind host (default: 127.0.0.1)
  --port PORT              Bind port (default: 8989)
  --prices PATH            Custom prices.json
  --openai-base-url URL    Upstream OpenAI-compatible API
  --anthropic-base-url URL Upstream Anthropic-compatible API
  --kill                   Activate the kill switch and exit
  --unkill                 Deactivate the kill switch and exit

Environment variables

Instead of CLI flags, you can set:

$env:ACB_OPENAI_BASE_URL = "https://api.openai.com"
$env:ACB_ANTHROPIC_BASE_URL = "https://api.anthropic.com"

Pricing file

Pricing is user-maintained. The bundled src/agent_circuit_breaker/prices.json contains a few common models:

{
  "gpt-4o": {
    "input_price_per_1m": 5.00,
    "output_price_per_1m": 15.00
  }
}

To override, copy the file, edit it, and run:

agent-circuit-breaker --prices "<path-to-prices.json>"

Get updated prices from:

• OpenAI pricing
• Anthropic pricing

---

📊 Accounting methodology

The proxy computes cost from token counts:

cost = input_tokens × (input_price / 1_000_000)
     + output_tokens × (output_price / 1_000_000)

Non-streaming requests

Reads usage from the JSON response. If usage is missing, falls back to estimating tokens from message length.

Streaming requests

Accumulates the provider's final usage SSE event:

• OpenAI: relies on stream_options.include_usage=true. If omitted, the proxy estimates output tokens from streamed content.
• Anthropic: uses message_start and message_delta usage events.

When a stream's cap is hit mid-flight, the proxy closes the upstream connection, emits an SSE error chunk, and ends the stream. No silent degradation.

Estimate label

Any cost calculated from on-the-fly token counts is stored in SQLite with estimated = 1 so you can distinguish exact vs. estimated spend.

---

📈 Performance

The breaker adds one local HTTP hop plus, on the response path, a durable SQLite spend-record write per request (that write is the point — it's what makes the cap survive a crash). To put a number on that fixed overhead, isolated from real provider latency, benchmarks/overhead.py drives an in-process OpenAI-compatible fake provider with N=200 non-streaming chat completions, measured two ways after a 25-request warmup:

Metric	Direct (us)	Through breaker (us)
p50 latency	11,618	33,799
p95 latency	14,183	44,810
p99 latency	15,640	49,390
mean latency	11,893	34,336

Methodology: the fake provider is an aiohttp server (stdlib http.server resets ~15% of rapid loopback connections on Windows, which would corrupt the measurement); the breaker runs as a real subprocess with its real SQLite state file, so the delta includes the process boundary and the per-request durable write. Both paths use one pooled httpx client, so the comparison is steady-state per-request cost, not connection setup. Reproduce with python benchmarks/overhead.py.

The ~22 ms / ~190% overhead is dominated by the per-request SQLite spend write, not the proxy's byte-forwarding logic. On a real remote provider whose own latency is in the hundreds of milliseconds, that fixed local cost is negligible; it only looks large here because the fake provider answers in microseconds. The breaker does not pool upstream connections, so each proxied request also opens one extra loopback connection to the provider — visible in the delta but immaterial against a real network upstream.

---

🧪 Verify the build

# Windows PowerShell
.venv\Scripts\python selfcheck.py

# Or with any Python that has the dependencies
python selfcheck.py

Expected output:

OK responses: 80
Blocked responses: 1
Kill switch ACTIVE for state: ...
Kill switch INACTIVE for state: ...
PASS

The test starts a fake provider with known usage, sends requests through the breaker until the run budget trips, and verifies both the cap and the kill switch.

---

🖼️ Visual summary

┌─────────────────────────────────────────────────────────┐
│  $ agent-circuit-breaker --run-budget 2.00              │
│  listening on http://127.0.0.1:8989                       │
│                                                         │
│  $ my-agent --loop                                      │
│  ...                                                    │
│  [agent-circuit-breaker] run budget $2.00 exhausted     │
│  [agent] 429 Budget exhausted                           │
│  Agent stopped. Total cost: $2.00                       │
└─────────────────────────────────────────────────────────┘

---

🔒 Trust model

What the breaker guarantees, and what it deliberately does not:

• Caps are enforced before forwarding, but spend is recorded after the response. The breaker checks the budget, then forwards; the request that crosses the cap is allowed through and is charged by the upstream provider. The cap stops subsequent requests, not the one that trips it. There is no way to know a response's cost before the provider returns it.
• Cost accuracy depends on your prices.json. Pricing is user-maintained and offline; if a model's price is missing or stale, the breaker estimates (and marks estimated = 1 in SQLite) or treats it as zero. The cap is only as honest as the prices file you keep current.
• Localhost-only by default; no auth. It binds 127.0.0.1 and has no authentication layer. Anyone who can reach the bound interface can drive spend through it. Don't bind it beyond loopback on a shared host without putting your own auth in front.
• No telemetry, no network except the upstream you configure. The only outbound calls are to the OpenAI-/Anthropic-compatible base URLs you pass. State lives in one SQLite file on disk.

---

⚠️ Limitations

• USD only — no currency conversion.
• Per-run + per-day budgets only — no per-model or per-tool caps in v1.
• Caps prevent future requests — the request that actually crosses the cap may still be charged by the upstream provider.
• Accurate streaming requires the provider to emit usage events. Otherwise the proxy estimates.

---

🤝 Contributing

This is intentionally small and sharp. Open issues or PRs for bugs and clear improvements. Large feature additions will be evaluated against the "single-purpose guard" philosophy.

---

📄 License

MIT — see LICENSE.

---

Made to save wallets from runaway agents.