costhelm

A self-hosted cost-governance gateway for LLM APIs. Your agents talk to one local HTTP endpoint; costhelm owns the provider keys and answers every call with the numbers that actually matter: what it cost, who spent it, and how much budget is left.

Most gateways treat cost as a report you read after the money is gone. costhelm treats it as a precondition: every call is projected at worst-case cost and admitted against a hard budget before any provider is contacted. A refusal is an HTTP 402 with the arithmetic in the body — limit, spend, projection, shortfall — and it costs $0, because nothing was sent.

call 4: ollama/phi4:latest — $0.017000 (total $0.047000) — 'Running budget checks before...'
call 5: REFUSED — session:budget-agent-demo would exceed its lifetime budget:
        spent $0.047000 of $0.050000, this call projects $0.016000

Features

Quickstart

uvx costhelm serve

(or from a clone: uv sync && uv run costhelm serve)

Add a provider key to .env (see .env.example) — or none at all if Ollama is running locally:

curl -s localhost:8111/v1/chat -H 'Content-Type: application/json' \
  -d '{"prompt": "hello", "max_tokens": 50}'

Every response carries cost, budget, cache, and router_decision envelopes. Open http://localhost:8111/ for the dashboard.

To see budget enforcement end-to-end, run the example agent — it arms a $0.05 ceiling, spends it, and prints the 402:

uv run python examples/budget_agent.py

How a request flows

POST /v1/chat
  │
  1. build_context      normalise, estimate tokens, resolve principal, agent pin
  2. semantic cache     hit → return stored answer, bill $0, record tokens saved
  3. select_candidates  router-LLM tier classify → role clamp → cost/quality order
  4. dispatch           ┌ cascade loop ──────────────────────────────┐
                        │  for each candidate:                       │
                        │    budget admission (worst-case, pre-call) │──402
                        │    provider call (+1 retry, backoff)       │
                        │    structured-output validate/repair       │
                        │    confidence check → maybe escalate tier ─┘
                        │    meter → priced ledger row, OTel span
                        └ return text + cost/budget/cache envelopes

Each stage is a module in costhelm/pipeline/, and each is inert unless configured: with the shipped config files, costhelm behaves like a plain failover gateway.

Configuration

Four YAML files, three-layer resolution — an env-var path override wins, then ~/.costhelm/<name>.yaml, then the packaged default. costhelm paths prints what resolved where.

Secrets and switches live in .env — see .env.example. A provider registers only if its key variable is set.

Client

from costhelm.client import CosthelmClient, BudgetExceeded

async with CosthelmClient() as gw:
    await gw.set_budget("session:run-42", 0.05, period="day")
    try:
        r = await gw.chat("hello", request={"session": "run-42", "auto_route": "worker"})
        print(r["text"], r["cost"]["total_usd"])
    except BudgetExceeded as e:
        print("refused:", e.envelope["shortfall_usd"])

The wire contract is plain HTTP JSON — any language works. The client holds no provider credential; that boundary is the point.

Deployment

deploy/ has systemd/launchd/Windows-service templates, a Modal wrapper, and a docker-compose file for a local Jaeger to receive the OTel spans. Docs: https://costhelm.com

Origins

costhelm grew out of an agentic-AI course project (a gateway serving a budget-aware agent runtime) and was extracted and restructured into a standalone framework.

License

MIT