mcp-budget-governor

Distributed, cost-denominated budget enforcement for MCP servers. Per-user quotas, per-tool limits, and a global spend circuit breaker, in atomic Redis counters that reset themselves at UTC midnight. Three lines to add to a server you already own.

⚠️ Pre-release. Not yet published to PyPI. See Status.

from fastmcp import FastMCP
from mcp_budget_governor import Governor, Limit, Policy, RedisBackend, Scope, Unit, Window, usd
from mcp_budget_governor.integrations.fastmcp import BudgetMiddleware

policy = Policy.of(
    Limit("per_user_calls", cap=2_000, window=Window.DAY, scope=Scope.USER),
    Limit("burst", cap=30, window=Window.MINUTE, scope=Scope.USER),
    Limit(
        "global_spend",
        cap=usd(25),
        window=Window.DAY,
        unit=Unit.USD_MICROS,
        breaker=True,
        gated=False,
    ),
)
governor = Governor(policy, RedisBackend.from_url("redis://localhost"))

mcp = FastMCP("my-server")
mcp.add_middleware(BudgetMiddleware(governor))

That's the whole integration. Under it, per call:

from mcp_budget_governor import Context

(await governor.check(Context(user="u_42"))).raise_for_status()  # admit or reject
result = await do_the_work()
await governor.meter_tokens(
    "global_spend",
    "claude-opus-5",  # charge what it cost
    input_tokens=3_200,
    output_tokens=850,
)

Why this exists

Agents call tools in bursts, at machine speed, with no human watching the bill. The failure mode is well documented and boring: a loop calls one tool a few hundred times in half an hour and burns most of a day's budget before anyone notices.

There are already two ways to deal with that, and a gap between them.

FastMCP's RateLimitingMiddleware is in-process, so with four replicas a cap of 100 quietly becomes 400. Gateways solve that by putting a proxy in front of everything, which is a large thing to adopt if all you wanted was a spending limit. This library is the middle: you keep your server, your transport, and your deployment, and you get counters that are correct when there is more than one of you.

And it counts money, not calls. A call cap is a poor proxy for a bill when one tool call costs a fraction of a cent and the next one costs a dollar.

Install

pip install mcp-budget-governor[redis,fastmcp]

Both are optional extras. Without redis you get an in-process backend that is correct for a single replica and useful in tests; without fastmcp you can still use the ASGI middleware, the @governed decorator, or the governor directly.

How it fits together

flowchart LR
    call(["tool call"]) --> ident["identify()<br/><i>your code — who is calling?</i>"]
    ident --> ctx["Context<br/>user · tenant · tool · session"]

    subgraph check["check() — before the work"]
        direction TB
        pick["applicable limits<br/><i>a limit whose scope the context<br/>can't supply is skipped</i>"]
        pick --> k["key = mcpbg:limit:scope:bucket<br/><i>bucket is a UTC timestamp;<br/>TTL runs to the end of it</i>"]
        k --> lua["consume — one atomic Lua script<br/>GET → compare to cap → INCRBY → repair TTL"]
    end

    ctx --> check
    check -->|any limit full| roll["roll back everything<br/>this call charged"]
    roll --> rej(["429 / 503 + Retry-After"])
    check -->|all fit| run["run the tool"]

    subgraph meter["meter() — after the work"]
        direction TB
        m["add the real cost<br/><i>ungated: always recorded</i>"] --> brk{"total past cap?"}
    end

    run --> meter
    meter -->|"yes, first time"| trip(["breaker trips —<br/>next caller is shed"])
    meter -->|no| ok(["result"])

    check -.->|backend unreachable| fm{"limit's<br/>fail_mode"}
    fm -.->|CLOSED| rej
    fm -.->|OPEN| loc["process-local counter<br/><i>per-replica, not none</i>"]
    loc -.-> run

    style rej fill:#7f1d1d,color:#fff
    style trip fill:#7f1d1d,color:#fff
    style ok fill:#14532d,color:#fff

Counters key on a UTC time bucket, so nothing ever resets one: tomorrow is a different key and yesterday's expires on its own. No cron, no reset job, and two replicas computing a bucket for the same instant agree by construction.

How you attach it

None of them can guess who is calling — transports authenticate differently and only your server knows how. Pass identify= to get per-user limits; without it you get per-tool.

Core ideas

Limits are declarative. A policy is a list. Adding a ceiling is adding a Limit, not editing a settings class, a middleware, and three clients.

Scopes compose. scope=Scope.USER is per user. scope=(Scope.USER, Scope.TOOL) is per user per tool — a separate counter, not a shared one. A limit whose scope the caller can't supply is skipped, so a per-tenant limit sits inert on a single-tenant deployment and activates the day you start passing a tenant.

Windows reset themselves. Every key ends in a UTC time bucket and carries a TTL to the end of it. Nothing resets a counter — tomorrow is simply a different key, and yesterday's expires on its own. No cron, no reset job, and two replicas computing a bucket for the same instant produce the same string, so they share a counter by construction.

Check and meter are different operations. You cannot know what a model call costs until it has finished, so a call is admitted against a quota and charged on completion:

(await governor.check(ctx)).raise_for_status()  # gated: refuse without charging
result = await run_the_tool()
await governor.meter("global_spend", cost_of(result))  # metered: always record

A refused call is never charged (otherwise retries inflate the counter and any usage reading built on it is fiction). A completed call is always charged, even if it lands over the ceiling (otherwise the spend counter under-reports real spend, which defeats having one). The call that crosses the line completes; the next one is shed.

Fail-open vs fail-closed is per limit. When Redis can't be reached, a limiter has to choose between availability and enforcement. This library makes each limit choose:

Limit("per_user_calls", cap=2_000, window=Window.DAY, fail_mode=FailMode.OPEN)
Limit("global_spend", cap=usd(25), window=Window.DAY, fail_mode=FailMode.CLOSED)

That split is the point. A per-user quota failing open risks one user's fair share. A spend breaker failing open risks the bill. The system this was extracted from failed open everywhere and said so in its own docs — "a Redis outage disables the per-minute limiter, the per-user daily quota, and the global breaker simultaneously" — which was a reasonable trade for the quota and a bad one for the breaker.

And fail-open doesn't mean unlimited. When Redis is unreachable, a fail-open limit is re-evaluated against a process-local counter instead of being waved through, so a cap of 100 across four replicas degrades to 400 rather than infinity. It's worse than working Redis and far better than nothing, and it can only ever reject calls plain fail-open would have allowed. Pass local_fallback=False for the old behaviour.

Money, not calls

A budget in dollars needs a price table, and a stale table is worse than none — it silently stops matching the bill. So the bundled one is opt-in and dated, and an unknown model raises instead of pricing at zero (a typo must not quietly disable the budget for that call path):

governor = Governor(policy, backend, prices=PriceTable.builtin())
await governor.meter_tokens("global_spend", "claude-opus-5", input_tokens=3_200, output_tokens=850)

Override anything that matters to you — a negotiated rate, a provider it doesn't cover, a price that moved since this release — with PriceTable.builtin().with_price("claude-opus-5", 4, 20).

Costs are integers in USD millionths, not floats. A budget that accumulates rounding error is a budget that disagrees with the invoice, and at $3/Mtok a single token is a number no float should be asked to add repeatedly.

Reserve and settle

Metering after the fact is cheap and right for most traffic: the ceiling is checked before the call, the real cost recorded after, and an overshoot is bounded by one call. That stops being true when a single call can be expensive and many run at once — ten concurrent callers all see an intact ceiling, all proceed, and the budget lands far past its cap with nobody at fault.

Reserving closes that window by charging an estimate before the work, so concurrent callers can see each other:

async with governor.reserved("global_spend", usd(0.05), ctx) as r:
    result = await call_the_model()
    r.actual = cost_of(result)  # settles to the real number on exit

Overshoot becomes bounded by the estimate's error rather than by how many calls are in flight. A reservation that is never settled — the process died — stays charged until its window rolls over, which is the safe direction: losing the charge would mean spending money the counter never saw.

Why Lua

Both mutating operations are single Lua scripts, because Redis runs a script to completion without interleaving another client. The obvious Python translation — GET, compare, INCR — is three round trips with two gaps, and under concurrency it lets N callers all read the same under-cap value and all increment past it.

That isn't theoretical, and the size of it is worth being precise about. benchmarks/ runs the same governor over both backends — 500 concurrent calls against a cap of 50:

The naive version enforced nothing at all: every caller read 0, every caller concluded it fit. At five cents a call that is $22.50 spent against a budget that had $2.50 left in it. The same claim is enforced as a test, not just measured here.

The scripts also repair a missing TTL on every write rather than setting one only on first write. A key whose EXPIRE was lost — process died between commands, failover dropped it — would otherwise live forever, and a day-bucketed counter that never expires never resets. That user is locked out until a human notices, which is a much worse failure than the dropped EXPIRE that caused it.

What it costs

Serial check() latency, 2,000 calls against a loopback Redis, in microseconds:

Roughly one round trip per gated limit — the cost is linear in how many ceilings a call has to clear, and a metered limit costs nothing until you meter it. At two gated limits that is ~0.5ms, which is 0.07% of an 800ms model call and a real fraction of a tool call that does nothing. Govern the calls that cost money; the ones that don't were never the problem.

Loopback Redis is the best case: a managed Redis in another AZ adds its RTT to every figure in that column, and that RTT will dominate. The shape transfers, the microseconds don't. Reproduce with uv run python benchmarks/bench.py --redis redis://localhost:6379/15; details and caveats in benchmarks/README.md.

Production provenance

This is extracted from the cost-control layer of a live application — an LLM chat backend (Groq) over three flight/hotel data providers, running per-user daily quotas and a global daily spend breaker across a FastAPI service and a Temporal worker sharing one Redis. The production ceilings are:

The breaker exists because the failure it defends against is silent. A leaked session or a runaway refresh loop doesn't page anyone — it just spends, at machine speed, until the invoice arrives weeks later. A per-user quota can't catch that, because the spend is spread across users and no single one of them looks abnormal. Only a global ceiling does.

What's here is that design, generalised: arbitrary limits instead of four hardcoded settings fields, cost units instead of raw token counts, and a per-limit answer to the Redis-outage question the original accepted as residual risk.

One bug is worth reporting because it came out of the extraction rather than the original. The first draft evaluated limits in policy order and returned on the first rejection — but a call rejected by a later limit had already been charged to every earlier one. With a daily quota declared before a per-minute burst cap, a user throttled by the burst cap silently drained their daily allowance on calls that never ran. check() is now all-or-nothing: a rejection rolls back everything it charged. The source system has the same latent behaviour and has never hit it, having only two ceilings that rarely both apply.

Python and TypeScript, one budget

Most MCP servers are TypeScript, so there is a sibling package — and the interesting part isn't parity. A Node server and a Python worker pointed at one Redis enforce one budget. They are two clients of a single enforcement layer, not two libraries that happen to behave alike.

That holds because the contract is neither language:

• The Lua scripts live in lua/ and are loaded from there by both packages. There is no translation to drift, because there is no translation.
• The key scheme is verified, not asserted. conformance/ generates keys, TTLs, buckets, USD conversions, price calculations, and script digests from the Python implementation; the TypeScript suite checks itself against them. CI regenerates them and fails if the committed ones are stale, so a Python-side change nobody mirrored breaks the build instead of passing against a fixture.
• A live cross-language test drives one Redis from both languages in the same test — a charge written by Python must be visible to TypeScript, and one cap must be enforced across both.

The TypeScript package also ships a Postgres backend (sql/mcpbg.sql) for deployments whose only shared store is Postgres: same five-method contract, atomicity from row locks instead of Lua, and counters that survive restarts — which neither the in-process backend nor a Redis-less deployment can otherwise get. The key scheme is what makes it work: a new window is a new key, so expiry is garbage collection rather than correctness.

The suite is sensitive enough that changing a single separator character in the TypeScript key builder fails it.

Status

Development

uv venv && uv pip install -e ".[dev,redis]"
uv run pytest              # unit suite, 95% coverage gate
uv run ruff check .
uv run mypy

cd ts && npm install && npm test   # the TypeScript sibling + conformance

Tests run against both backends via the same parametrised suite — the in-memory backend's only justification is being a faithful stand-in for Redis, so the suite enforces the equivalence rather than assuming it. The Redis tests use fakeredis, which executes the Lua for real; a mocked backend would pass while the scripts were nonsense. CI additionally runs the whole suite against a real Redis service container, because fakeredis's Lua is an implementation of Redis's, not Redis's.

Contributions welcome — see CONTRIBUTING.md.

License

MIT