Baar-Core

Pre-flight runtime governance for LLM agents. Enforces hard execution limits locally — before requests reach the provider.

Stop LLM API calls before they happen. Not after.

pip install baar-core

I left an agent loop running overnight. Woke up to a $47 bill — 20,000 GPT-4o tokens answering "what time is it?" queries.

Baar-Core would have stopped it at $0.10. Before the first overage call. No network request made. $0 spent.

from baar import BAARRouter

router = BAARRouter(budget=0.10)          # hard cap: $0.10 total
router.chat("What time is it?")           # → cheap model, ~$0.0001
router.chat("Write a CUDA matmul kernel") # → capable model if budget allows
# budget exhausted → raises BudgetExhausted, zero API calls made

84–94% cost reduction in benchmarks — see benchmarks

Built for

• Teams deploying autonomous agents in production
• SaaS founders giving users LLM access with per-user quotas
• Anyone who has had (or fears) a runaway agent bill

The problem with every other solution

Most cost tools track spend after the fact. You get an alert when the bill is already large.

LiteLLM's budget manager, Portkey rate limits, provider spend alerts — they all tell you what happened. They don't stop it mid-flight.

Helicone observes the disaster. Portkey rate-limits after the fact. Baar prevents it.

Baar-Core is a local kill-switch. Before each call, it estimates the cost. If the remaining budget is too low, it raises an exception locally — no DNS lookup, no TCP connection, no token consumed. The call never leaves your machine.

How it works

User task
    │
    ▼
┌─────────────────────────────────┐
│  Pre-flight budget check        │ ← if estimated cost > remaining budget
│  (local, zero network)          │   raise BudgetExhausted immediately
└────────────┬────────────────────┘
             │ affordable
             ▼
┌─────────────────────────────────┐
│  Semantic complexity router     │ ← cheap LLM scores complexity 0.0–1.0
│  (gpt-4o-mini, ~$0.000015/call) │
└────────────┬────────────────────┘
             │
      ┌──────┴───────┐
      │              │
   simple         complex
      │              │
      ▼              ▼
 Cheap model    Budget check
 (fast, $)      ├─ affordable → Capable model ($$$)
                └─ too close  → Downgrade to cheap model ($)

1. Pre-flight check — Estimates cost locally before any network call. Kills the request if it would overshoot.
2. Semantic routing — A fast, cheap model scores task complexity. Not keyword matching — actual semantic understanding.
3. Budget-aware downgrade — Running low? Hard tasks automatically fall back to the cheaper model so the turn still completes.

Quick start

from baar import BAARRouter, BudgetExhausted

# Basic usage
router = BAARRouter(budget=0.10)
reply = router.chat("Explain recursion with a Python example")
print(reply)
print(f"Spent: ${router.spent:.5f} / Remaining: ${router.remaining:.5f}")

# Multi-step with a report
log = router.run([
    "What is 42 * 17?",
    "Translate 'good morning' to Japanese",
    "Design a distributed rate-limiter for 100k RPS — include trade-offs",
    "Convert 72°F to Celsius",
])
log.print_report()

# Async
import asyncio
async def main():
    router = BAARRouter(budget=0.05)
    reply = await router.achat("Summarize the CAP theorem")
    print(reply)

asyncio.run(main())

# Kill-switch in action
router = BAARRouter(budget=0.00001)
try:
    router.chat("Any prompt at all")
except BudgetExhausted as e:
    print(f"Blocked before API call. Remaining: ${e.remaining:.6f}")
    # Zero network calls made. $0 spent.

Works with any LiteLLM-supported provider: OpenAI, Anthropic, Groq, Together, Ollama, OpenRouter, Azure, and more.

LangChain & LangGraph integration

pip install baar-core[langchain]

Pattern 1 — add a kill-switch to an existing chain (zero refactoring):

from baar import BAARRouter
from baar.integrations.langchain import BaarCallbackHandler
from langchain_openai import ChatOpenAI

router  = BAARRouter(budget=0.05)
handler = BaarCallbackHandler(router)

llm   = ChatOpenAI(model="gpt-4o", callbacks=[handler])
chain = prompt | llm | StrOutputParser()
# BudgetExhausted is raised before the API call when budget is too low

Pattern 2 — BaarChatModel as a drop-in ChatModel (full routing + downgrade):

from baar.integrations.langchain import BaarChatModel

router = BAARRouter(budget=0.10, small_model="gpt-4o-mini", big_model="gpt-4o")
llm    = BaarChatModel(router=router)

chain = prompt | llm | StrOutputParser()   # works anywhere ChatOpenAI would

Pattern 3 — LangGraph agent with budget protection:

from langgraph.prebuilt import create_react_agent

agent = create_react_agent(model=llm, tools=[...])
# every tool-call iteration is budget-checked before the LLM is invoked
# BudgetExhausted stops the loop locally — no dangling API calls

Full example: langchain_guardrail.py

LangGraph middleware

BaarMiddleware wraps any compiled LangGraph graph and enforces hard limits at the outer loop level — before a single node executes.

from baar import BAARRouter
from baar.integrations.langgraph import BaarMiddleware, StepLimitExceeded

router = BAARRouter(budget=0.50)
graph  = workflow.compile()          # your compiled LangGraph graph

protected = BaarMiddleware(
    graph=graph,
    router=router,
    max_steps=10,        # stop after 10 outer invocations
    max_budget=0.05,     # block if less than $0.05 remaining
    recursion_limit=25,  # injected into LangGraph config automatically
)

try:
    result = protected.invoke({"messages": [...]})
    print(f"Steps used: {protected.steps}/10")
except StepLimitExceeded as e:
    print(f"Stopped at {e.steps}/{e.max_steps} steps")
except BudgetExhausted as e:
    print(f"Budget gate: ${e.remaining:.6f} remaining")

Works with all four call modes: invoke, stream, ainvoke, astream.

Full example: langgraph_middleware.py

Tool execution guards

@baar_guard wraps any tool function — sync or async — with hard call limits and budget gates. Works with Claude, OpenAI/Codex, LangChain, LangGraph, or plain Python. The AI never knows the guard is there.

from baar.integrations.tools import baar_guard, ToolCallLimitExceeded

@baar_guard(router=router, max_calls=5, cost_per_call=0.001)
def web_search(query: str) -> str:
    return search_api(query)

@baar_guard(router=router, max_calls=3, min_budget=0.02)
async def generate_image(prompt: str) -> str:
    return await image_api(prompt)

• max_calls — hard call count limit per session. None = unlimited.
• min_budget — block if router.remaining drops below this threshold.
• cost_per_call — deduct a fixed amount from budget on each successful call. No charge on failure.

Catch limit exceptions in your agent dispatcher so the AI gets a readable error instead of a crash:

try:
    result = web_search(query)
except ToolCallLimitExceeded as e:
    result = f"Tool '{e.tool_name}' limit reached ({e.calls}/{e.max_calls})."
except BudgetExhausted as e:
    result = f"Budget too low. Remaining: ${e.remaining:.6f}."

Works with OpenAI Codex CLI — set openai_base_url = "http://localhost:8000/v1" in ~/.codex/config.toml to route all Codex calls through Baar.

Full examples: tool_guard.py · claude_tool_guard.py · openai_tool_guard.py

Loop detection

A budget stops a looping agent once the money is gone. LoopDetector stops it after a few wasted calls. It watches what each tool call returned, not just that it was called, so an agent repeating a call that keeps giving the same answer is stopped while the same call making progress (a job going queued → running → done, a search with a new cursor) is left alone.

from baar import BAARRouter, LoopDetector, LoopDetected
from baar.integrations.tools import baar_guard

router = BAARRouter(budget=0.50)
loops = LoopDetector(on_loop=lambda e: log.warning("possible loop: %s", e))  # one per run

@baar_guard(router=router, loop_detector=loops)
def web_search(query: str) -> str:
    return search_api(query)

@baar_guard(router=router, loop_detector=loops)
def read_file(path: str) -> str:
    return open(path).read()

try:
    result = web_search(query)
except LoopDetected as e:
    result = f"Stopped: '{e.tool_name}' {e.pattern} loop ({e.count}x). Try a different approach."

It refuses the call that would continue a loop before it runs. One step earlier it tells the model: on the first repeat, baar_guard adds a note to the tool's string result ([baar] The tool 'web_search' has returned the same result 2 times… Change the arguments, use a different tool, or answer with what you already have.), and on_loop gets a warning. In a live benchmark, the note was what mattered: the model changed course in 19 of 20 runs, while stopping without telling it did worse than no detection at all. Pass loop_nudge=False to baar_guard to leave results untouched; non-string results are never changed.

Three patterns:

• Waiting is not looping: a short result (≤ 200 characters) saying running, pending, queued, in progress, processing, booting, starting, initializing, waiting or not ready never counts, so polling a job isn't stopped. in_progress=re.compile(...) changes the words; in_progress=False counts them.
• exempt_tools=["job_status"] — never count a tool that legitimately returns the same thing many times.
• ignore_args=["request_id", "timestamp"] — drop volatile arguments, at any depth, before comparing.
• normalize_result=fn — map a result before hashing, e.g. to drop a timestamp it carries.
• loops.tighten() — lower every threshold by one after summarizing the agent's history (summarization roughly doubles looping, per DeLoop).
• Not a framework user? Call loops.before(tool, args) and loops.after(tool, args, result=...) (or error=exc) yourself.

What it can't catch: a loop that rephrases itself every step (new wording, same empty result) never repeats exactly. Keep a run budget for those: the two together stop every loop in the eval below without stopping a healthy run.

Loop eval

python benchmarks/loop_eval.py replays 180 labeled, deterministic agent traces (no API keys): 75 loops (exact repeats, error retries, ping-pong, repeats with a changing request id, rephrased) and 105 healthy runs built to look like loops (polling, pagination, rate-limit retries, a flaky API failing on and off, re-reading a changed file, long research). Loops run until something stops them; the run budget is sized above the longest healthy run.

max_calls wastes less money only by also stopping 41% of healthy runs (pagination, long research, the flaky API). Counting calls without their results stops every flaky-API run; the detector stops none, because errors only count in a row. detector+budget is how you'd ship it: the detector plus a $0.40 run budget. The 6-identical-running polls are no longer stopped even without exempt_tools (they were 14.3% false stops before 0.9). Overhead is about 10 µs per tool call. The headline holds for every generator seed from 1 to 50 (100% caught, 0% false stops, median 2 wasted calls). Per-category results: python benchmarks/loop_eval.py.

MCP server (Claude Code / Cursor)

examples/mcp_server.py exposes three budget-guarded tools to any MCP client — no API key required on the client side.

# pip install baar-core mcp firecrawl-py

from baar import BAARRouter
from baar.integrations.tools import baar_guard

router = BAARRouter(budget=0.10, use_llm_router=False)

@baar_guard(router=router, max_calls=20, cost_per_call=0.001)
async def web_search(query: str) -> str: ...

@baar_guard(router=router, max_calls=10, min_budget=0.005)
async def run_python(code: str) -> str: ...

Register with Claude Code — add to ~/.claude/claude.json:

{
  "mcpServers": {
    "baar": {
      "command": "python3",
      "args": ["/path/to/examples/mcp_server.py"],
      "env": { "BAAR_BUDGET": "0.10" }
    }
  }
}

Claude Code will show get_budget_status, web_search, and run_python as tools automatically. Every call is budget-gated — the session stops spending the moment the cap is hit.

OpenAI-compatible HTTP server (Vercel AI SDK, LlamaIndex, curl)

pip install baar-core[vercel]

Wraps BAARRouter as a /v1/chat/completions endpoint — any OpenAI-compatible client works without code changes.

from baar import BAARRouter
from baar.integrations.vercel import create_app
import uvicorn

router = BAARRouter(budget=0.10)
app    = create_app(router, api_key="your-secret")  # api_key optional

uvicorn.run(app, host="0.0.0.0", port=8000)

Vercel AI SDK (TypeScript):

import { createOpenAI } from '@ai-sdk/openai';
import { streamText } from 'ai';

const baar = createOpenAI({ baseURL: 'http://localhost:8000/v1', apiKey: 'your-secret' });

const { textStream } = streamText({
    model: baar('baar'),
    messages: [{ role: 'user', content: 'Hello!' }],
});

LlamaIndex (Python):

# pip install llama-index-llms-openai
from llama_index.llms.openai import OpenAI as LlamaOpenAI
from llama_index.core import Settings

llm = LlamaOpenAI(
    model="baar",
    api_base="http://localhost:8000/v1",
    api_key="your-secret",
)

Settings.llm = llm  # drop-in for any LlamaIndex agent, RAG pipeline, or query engine
response = llm.complete("Explain the GIL in Python")

Budget errors surface as standard HTTP codes — 402 when the budget is exhausted, 422 when the value gate rejects the task. Streaming errors are delivered inside the SSE stream so the connection stays clean.

Real-world examples

Multi-tenant & per-user budgets

Give each user their own hard quota — enforced locally, persisted across restarts, no proxy or external service required.

from baar import BAARRouter, BudgetExhausted
from baar.core.stores import SQLiteBudgetStore

def router_for(user_id: str) -> BAARRouter:
    return BAARRouter(
        budget=0.10,
        store=SQLiteBudgetStore("budgets.db", namespace=user_id),
    )

# Alice and Bob share one database file — budgets are fully isolated
alice = router_for("alice")
bob   = router_for("bob")

alice.chat("Summarise this document")  # deducted from Alice's $0.10 only
bob.chat("Translate to French")        # Bob's quota untouched

Namespace isolation is enforced at the store level — one user exhausting their quota cannot affect another's. Concurrent writes across threads and across OS processes (gunicorn/uvicorn workers) are safe: SQLiteBudgetStore uses WAL mode and an exclusive transaction for the check-and-reserve step, eliminating the TOCTOU race that affects most multi-tenant budget implementations.

SaaS pattern — return HTTP 402 when a user's quota is exhausted:

@app.post("/chat")
async def chat(req: ChatRequest):
    try:
        reply = await router_for(req.user_id).achat(req.message)
    except BudgetExhausted as e:
        raise HTTPException(status_code=402, detail={
            "error": "quota_exhausted",
            "remaining_usd": e.remaining,
        })
    return {"reply": reply}

No billing service. No proxy. One SQLite file. Full example: fastapi_per_user_budget.py

Time-windowed quotas — scope budgets to a day or month by encoding the period in the namespace:

import datetime
period = datetime.date.today().strftime("%Y-%m")   # "2026-05"
store  = SQLiteBudgetStore("budgets.db", namespace=f"user:{user_id}:{period}")
# quota resets automatically next month — no cron job needed

Single-process scripts — JSON file store, no SQLite needed:

from baar.core.stores import FileBudgetStore

router = BAARRouter(budget=1.00, store=FileBudgetStore("my_budget.json"))

Benchmarks

Routing cost benchmark — mock mode

Mock mode runs the full routing pipeline with simulated completions to measure routing overhead and cost allocation without spending real money. Use this to tune thresholds before a live run.

Note: Accuracy figures in mock mode reflect simulated task responses, not real model capability. Use live mode for accuracy measurement. The cost figures and routing split percentages are the meaningful outputs here.

baar-bench --dataset all --limit 200 --budget 10 --mock \
  --complexity-threshold 0.80 --coding-threshold 0.75 --seed 42

HumanEval routes fewer tasks to the cheap tier because coding questions score high complexity — the router correctly identifies them as hard.

Live benchmark — real API calls (10 tasks per dataset)

baar-bench --dataset all --limit 10 --budget 2 \
  --complexity-threshold 0.80 --coding-threshold 0.75 --seed 42

Run it yourself: pip install baar-core datasets then baar-bench --limit 10 --mock (free) or add your API key for live results.

vs. alternatives

The key difference: every alternative routes and tracks. Baar-Core prevents — the exception is raised before a single byte leaves your machine.

Output limits

The pre-flight check covers the prompt, but the response is where cost can run away: one long answer, or a reasoning model's hidden thinking tokens, can cost more than the estimate. on_output_overrun decides how Baar handles that:

router = BAARRouter(budget=0.10, on_output_overrun="preflight", max_output_tokens=2000)

# or per call
router.chat("Write the full report", on_output_overrun="truncate")

* Prompt tokens are an estimate (exact for OpenAI text; approximate for Claude, Gemini, images and tool schemas), so these modes price the prompt with a 20% margin (prompt_margin=0.2). An undercount larger than that can still overshoot slightly.

preflight is most useful with your own max_output_tokens: without one, the worst case is the model's full max output (16k+ tokens), which blocks more calls near the end of a budget. Or let Baar learn it: overrun_risk=0.05 reserves the length that 95% of this model's responses have stayed under (a split-conformal quantile of the last 500 completed calls), instead of the worst case. max_tokens stays the full limit, so nothing is cut off; at most about 5% of calls can run past the reservation. Until a model has 20 completed calls, the worst case is reserved. Share what's learned between routers with output_lengths=OutputLengths(). The limit is always capped by the room left in the model's context window. Your own max_output_tokens is never sent above the model's real max output. In truncate mode, max_call_share=0.5 stops one call from reserving more than half of what's left, so concurrent calls near the cap still get through.

Calibrated mode: a guarantee per run, not per call

overrun_risk=r bounds each call separately, so a run of many calls can still overspend more often than r (2/30 runs in the benchmark below). calibrated bounds the whole run: pick a tolerance ε, and Baar picks how much output to reserve so that at most ε of your agent runs overspend, with confidence 1 − δ. Calibrate once on runs of your own workload:

from baar import BAARRouter, QuantileTable, ltt_select

table = QuantileTable.from_records(logged_calls)       # dicts with model, completion_tokens

def run_batch(beta, n):                                 # run n agent tasks with this beta,
    ...                                                 # return how many overspent

res = ltt_select([0.01, 0.05, 0.1, 0.2, 0.3, 0.5], run_batch, n=1000, eps=0.05, delta=0.05)
router = BAARRouter(budget=0.10, on_output_overrun="calibrated",
                    output_quantiles=table, calibration_beta=res)

ltt_select tests the levels from most to least conservative with exact binomial p-values (Learn-then-Test) and stops at the first it cannot certify; if none is certified, calibrated reserves the worst case. The guarantee assumes future runs look like the calibration runs. Method and results: research/.

In truncate mode the big model is used only if its full response fits, so models with a very large max output (GPT-5, o-series) will usually downgrade unless you set max_output_tokens.

Capping hidden reasoning. thinking_budget=4000 limits a reasoning model's thinking without touching the visible answer:

router = BAARRouter.anthropic(budget=0.50, on_output_overrun="preflight",
                              thinking_budget=4000, max_output_tokens=2000)

On Claude this also makes preflight practical: the worst case drops from the model's full 64k–128k output tokens to answer + thinking. Minimum thinking_budget is 1024 (Anthropic's floor); set it per call with router.chat(..., thinking_budget=2048). This is the split from Elastic Reasoning (arXiv 2505.05315): thinking gets its own budget and the answer keeps a reserved one.

Streaming calls record the provider's reported usage, including hidden reasoning tokens. If you stop reading a stream early, Baar closes it and, in preflight / truncate, records the reserved output rather than just the text you read. The value gate (value_fn) compares a task's value with its expected cost, not the worst case these modes reserve.

OverThink eval

python benchmarks/overthink_eval.py replays an agent run under the OverThink attack (arXiv 2502.02542): decoys hidden in what the model reads multiply its hidden reasoning (×10–46 here, up to ×46 in the paper) while the answer looks normal. 150 tasks, 30% poisoned, a $5.00 budget, a simulated Claude-class reasoning model ($3 / $15 per million tokens, 64k max output). Baar's real pricing and limits run unmodified; only the provider is simulated, and it honours max_tokens and the thinking budget like real APIs do. Seed 7:

Over seeds 1–30: post-hoc alerts overshoot in 30/30 runs and the default estimate in 27/30 (up to $0.66). preflight, preflight + thinking and truncate never overshoot, and preflight never cuts an answer off. overrun_risk=0.05 overshoots in 2/30 runs (up to $0.16) — the risk it accepts in exchange for using more of the budget. Capping thinking finishes every task because poisoned tasks stop being expensive; the simulation assumes the answer is unaffected by the cap, which real models don't guarantee. If you expect adversarial content, use preflight (or truncate) and set a thinking_budget on reasoning models.

From logs to a certificate in one command

The fastest way to use either certified feature: let the router log whole runs, then run baar-calibrate.

1. Log runs with a budget that does not bind (so every run shows what it would have cost), one router per run:

from baar import BAARRouter

router = BAARRouter(budget=100.0, run_log_path="runs.jsonl")
with router.recording_run() as run:
    ...                                  # your agent loop: router.chat(...), router.record_event("tool_error")
    run.finished = True                  # the agent gave a final answer
    run.resolved = tests_pass            # optional: was it correct?

2. Certify a budget and a risk level. It needs at least ⌈ln δ / ln(1 − ε)⌉ calibration runs (59 at ε = δ = 0.05) and tells you how many more to log if you have too few:

baar-calibrate runs.jsonl --budget 0.50 --eps 0.05                    # calibrated reservation level
baar-calibrate runs.jsonl --budget 0.50 --eps 0.05 --early-stop \
               --harm resolve --counter tool_error -o early_stop.json  # certified early stopping

3. Use it, and watch that it keeps holding:

from baar import BAARRouter, Certificate, GuaranteeMonitor

cert = Certificate.load("baar_cert.json")
monitor = GuaranteeMonitor.for_certificate(cert)       # share one monitor across runs
router = BAARRouter(budget=0.50, guarantee_monitor=monitor, **cert.router_kwargs())
...
router.finish_run(finished=True)                       # or use router.recording_run()

The monitor raises a GuaranteeAlarm warning (or calls on_alarm) when live runs show that more than ε of them overspend, which happens when the agent, its prompts or its tasks have changed since calibration. Its test is anytime-valid: checking after every run keeps the false-alarm probability below alpha. The same steps are available in Python as load_runs, certify_calibrated and certify_early_stop.

Certified early stopping

Long agent runs that will blow their budget are otherwise refused only once the money is gone. An EarlyStopPolicy checks the run at fixed checkpoints and stops it before the next call when it is predicted to exceed the budget, with a certificate: at most ε of all runs are stopped although they would have finished within budget (confidence 1 − δ). As a share of the runs that would have finished, the rate is higher by the inverse of how common they are. Fit on one set of logged runs, certify on another:

from baar import BAARRouter, fit_early_stop, calibrate_early_stop, RunStopped

policy = fit_early_stop(train_runs, budget=5.0, counter_keys=["tool_error"])
policy = calibrate_early_stop(policy, calibration_runs, eps=0.05, delta=0.05)
router = BAARRouter(budget=5.0, early_stop=policy)
router.record_event("tool_error")   # report behaviour as the run goes; RunStopped ends the run early

On 3,000 real coding-agent runs, at ε = 0.05 this saved 11–23% of all spend when the protected runs are those that would solve their task (harm="resolve"), and 4.5–16% when protecting every run that would finish (harm="finish", the default), depending on how tight the budget is. Fit the model on runs disjoint from the calibration runs and fix the budget before calibrating; the guarantee assumes future runs look like the calibration runs. See research/.

Budget awareness and wind-down

A kill-switch alone stops an agent mid-task with nothing to show for the money spent. Baar can also let the agent see its budget, and end with an answer instead of an exception. (BATS, arXiv 2511.17006: a short budget status block each step makes agents spend better, with no training.)

from baar import BAARRouter, WindDown, BudgetExhausted

router = BAARRouter(
    budget=1.00,
    budget_awareness=True,                        # or a formatter: lambda status: "..."
    wind_down=WindDown(at=0.9, reserve=0.05),     # final call past 90% used / 5% left
)

try:
    while True:
        reply = router.chat(next_step())
except BudgetExhausted as e:
    answer = e.partial_result                     # the final, wrapped-up answer

• router.budget_status() → BudgetStatus(total, spent, remaining, utilization, calls, avg_call_cost, calls_left, winding_down). LangChain's BaarChatModel, CrewAI's BaarCrewLLM and LangGraph's BaarMiddleware expose it as .budget_status.
• budget_awareness=True adds [budget] $0.3200 of $1.00 used (32%) · ~17 calls left at the current rate to the end of each call's last user message — not the system prompt, so a cached prompt prefix stays intact.
• WindDown: once utilization reaches at, or only reserve of the budget is left, the next call is the final one — it carries a wrap-up instruction (instruction= to change it). Every later call raises BudgetExhausted locally with that answer in .partial_result, until the budget is reset or a new window starts. reserve decides when to wind down; it doesn't fence money off from earlier calls, so pair it with on_output_overrun="preflight" if the final call must not overshoot.
• Policy: Rule(when={"utilization": ">= 0.8", "plan": "free"}, then="wrap_up") makes a call the final one.

Prompt caching

Providers bill a prompt prefix they've seen recently at a discount (OpenAI and DeepSeek automatically, Anthropic with cache_control, Gemini implicitly) — in an agent loop that's most of every prompt. Baar reads the cached tokens each call reports (StepResult.cached_prompt_tokens), records streams at the cache rate, and reports the hit rate (log.summary()["cache_hit_rate_pct"], and in print_report()). Reservations still use the uncached price, so caching never weakens the budget guarantee.

Does routing still pay when switching models throws the cache away? python benchmarks/cache_eval.py replays 24 conversations × 12 turns with 2k–80k-token contexts, real gpt-5 / gpt-5-mini pricing, and a provider that caches like OpenAI (per model, prefix ≥ 1024 tokens). 30% of turns are hard. Seed 7:

Routing saves 31% even after the cache losses: each model keeps its own cache, and gpt-5-mini is about 5× cheaper on every line (input, cached input and output). We also tried "sticky" routing — keeping an easy turn on gpt-5 whenever its cached prefix made that turn cheaper — and it cost more overall ($2.34): rebuilding the small model's cache once pays for itself within a turn or two, which a per-turn rule can't see. So Baar doesn't offer it.

Security

Baar-Core maps to OWASP LLM10:2025 — Unbounded Consumption. The pre-flight kill-switch is a direct mitigation for Denial-of-Wallet attacks: even if an adversary crafts a prompt designed to trigger expensive model calls, the local budget cap catches it before any provider request is made.

Details: RESEARCH.md

Configuration

router = BAARRouter(
    budget=0.10,                    # hard cap in USD
    small_model="gpt-4o-mini",      # cheap tier (any LiteLLM model)
    big_model="gpt-4o",             # capable tier
    complexity_threshold=0.80,      # 0.0–1.0: higher = more traffic to cheap model
    min_cost_threshold=0.0001,      # kill-switch floor — reject if any call costs more
    routing_task_char_limit=500,    # chars sent to routing LLM (head+mid+tail sample)
    use_llm_router=True,            # False = rule-based heuristic only (no routing cost)
    small_fallback_models=["gpt-4o-mini-2024-07-18"],  # failover chain
    big_fallback_models=["gpt-4o-2024-08-06"],
    telemetry_jsonl_path="telemetry.jsonl",  # optional audit log
)

Budget pressure — as spend approaches the cap, the effective complexity threshold rises automatically. The big model becomes harder to justify as you run low, so more traffic shifts to cheap naturally.

Telemetry — inspect spend, routing splits, and reject rates:

baar-telemetry telemetry.jsonl

Resilience testing — adversarial scenarios (complexity games, tight budgets, padding attacks):

baar-stress

License

MIT — LICENSE