Ballast

A resilience and cost-control plane for multi-agent AI systems.

Landing page & interactive demo → — kill a simulated API and watch the breaker trip, reroute, and recover, right in your browser.

The dashboard during a real injected incident: breaker trips in milliseconds, fallbacks serve while it's open, half-open probes recover it — every event in the live feed.

Ballast sits between your agent orchestrator (LangGraph, CrewAI, AutoGen, custom loops) and everything it depends on — LLM APIs, tools, databases. It detects overload and failure in real time, absorbs traffic spikes with backpressure, and falls back to cheaper alternatives instead of hard-failing. A chaos-injection mode lets you prove your pipeline survives an outage instead of hoping it does — and a live dashboard shows the whole story as it happens.

Agent orchestrator (LangGraph / CrewAI / custom)
        │
        ▼
 ┌─────────────────────────────┐
 │   Ballast interceptor SDK   │   ← @guarded decorator / guard() context manager
 └─────────────────────────────┘
        │
        ▼
 ┌─────────────────────────────┐
 │  Backpressure controller    │   ← global; FIFO queue + shedding
 └─────────────────────────────┘
        │
        ▼
 ┌─────────────────────────────┐
 │  Circuit breaker (per dep)  │   ← rolling health window per dependency
 └─────────────────────────────┘
        │
   ┌────┴─────┐
   ▼          ▼
Healthy    Fallback router
path       cache → cheaper model → static
   │          │
   └────┬─────┘
        ▼
 Real dependency call  ←──  chaos injector (opt-in fault injection)
        │
        ▼
 Event bus → dashboard · SQLite audit log

Why Ballast exists

Multi-agent systems ship to production with almost none of the hardening that microservices learned over 15 years:

• Cascading failure — one slow or failing tool piles up retries until the whole pipeline dies.
• No backpressure — a fan-out of 50 agent spawns hammers a downstream API with nothing throttling it.
• No graceful degradation — a dependency outage either hard-fails the run or silently corrupts it.
• Uncontrolled cost — agent swarms burn token budgets with no cost-aware routing.
• Unproven resilience — nobody can test whether the pipeline actually survives an outage before one happens.

Ballast ports the proven answers — circuit breakers, backpressure, graceful degradation, chaos engineering — to the agent world, as a drop-in layer that wraps your existing calls. No orchestrator rewrite.

Features

Quickstart

pip install ballast-agents                # core: zero runtime dependencies
pip install "ballast-agents[dashboard]"   # + FastAPI dashboard

(The distribution is ballast-agents; the import name is ballast.)

Wrap a call and configure your thresholds:

import ballast
from ballast import guarded

ballast.configure(
    dependencies={
        "openai_api":  {"failure_threshold": 0.5, "latency_multiplier": 3, "cooldown_s": 10},
        "postgres_db": {"failure_threshold": 0.3, "latency_multiplier": 2, "cooldown_s": 5},
    },
    max_concurrency=100,        # global in-flight ceiling
    max_queue_depth=500,        # waiting requests beyond it queue up to here
    budget_usd_per_hour=10.0,   # soft ceiling at 80%, hard at 100%
)

@guarded(
    dependency="openai_api",
    timeout=10,
    cache_ttl_s=300,                       # rung 1: serve recent identical answers
    cheaper=call_small_model,              # rung 2: downgrade simple prompts
    fallback="Service degraded — cached summary unavailable.",  # rung 3: static
    cost_fn=lambda r: r.usage_cost_usd,    # meter spend against the budget
)
def call_llm(prompt: str) -> str:
    return client.chat.completions.create(...)

with ballast.guard("postgres_db"):          # context-manager form
    rows = db.query(...)

Async works out of the box — @guarded detects async def and never blocks the event loop; timeout becomes a real cancelling timeout in async:

@guarded(dependency="openai_api", timeout=10, fallback="degraded")
async def call_llm(prompt: str) -> str:
    return await client.chat.completions.create(...)

async with ballast.aguard("postgres_db"):   # async context-manager form
    rows = await db.query(...)

Reference integration: examples/langgraph_example.py — a real LangGraph pipeline surviving a total LLM outage with zero failed invocations (the graph code itself is untouched; the integration is two decorators). With fallback_on_error=True, the fallback chain also serves during the breaker's detection window, so callers never see the outage at all.

When openai_api degrades, its breaker trips within a handful of calls; while it's open, requests are served by the fallback chain instead of erroring, and the breaker probes its way back to closed once the API recovers. Nothing else in your pipeline changes.

See it break (on purpose)

python examples/demo.py

50 concurrent workers, an 85%-failure chaos injection at t+3s, and a printed play-by-play:

[  3.03s] >> CHAOS INJECTED   mock_api {'kind': 'failure', 'value': 0.85, 'duration_s': 4.0}
[  3.06s] !! BREAKER TRIP     mock_api {'reason': 'failure_rate', 'failure_rate': 0.55}
[  3.06s] -> FALLBACK ACTIVE  mock_api (serving cached responses)
[  5.06s] ?? HALF-OPEN        mock_api
[  5.06s] !! BREAKER TRIP     mock_api {'reason': 'probe_failure', 'cooldown_s': 4.0}
[  9.06s] ?? HALF-OPEN        mock_api
[  9.06s] << CHAOS CLEARED    mock_api {'reason': 'expired'}
[  9.11s] OK BREAKER CLOSE    mock_api {'reason': 'probes_succeeded'}

calls: 10,466 | live: 4,553 | fallback: 5,901 | errors: 12
detection: breaker tripped 0.02s after chaos began (target: < 2s)

The dashboard

docker compose up --build          # → http://localhost:8080, zero setup
# or, without Docker:
pip install -e .[dashboard]
python -m ballast.dashboard        # → http://127.0.0.1:8080

Click Run demo scenario and watch a live incident: breaker cards flip CLOSED → OPEN → HALF-OPEN with cooldown countdowns, the fallback badge lights up, traffic and cost charts stream, and the chaos banner counts down with a one-click Stop now. The demo ships in-process — no API keys, no external services.

The dashboard binds to localhost and has no authentication — it is a local operations view, not an internet-facing service.

Chaos engineering

Chaos is opt-in twice: it only functions when BALLAST_CHAOS=1 is set or configure(chaos_enabled=True) is passed, and every injection is time-boxed (hard cap: 600 s — no indefinite faults). Faults wrap the real dependency call, so the breaker logic being exercised is the production code path, not a mock.

ballast.chaos.inject_failure("openai_api", rate=0.8, duration_s=30)   # random ChaosError
ballast.chaos.inject_latency("openai_api", multiplier=3.0)            # 3× slowdown
ballast.chaos.inject_corruption("openai_api", rate=0.5)               # mangled responses
ballast.chaos.scenario("api_outage").run("openai_api", duration_s=30) # preset bundle
ballast.chaos.clear()                                                 # stop everything

Presets: api_outage (100% failure), slow_api (3× latency), flaky_api (50% failure). Injected failures raise a distinct ChaosError, so logs and tests can never confuse injected faults with real ones.

Configuration reference

Per-dependency breaker settings (all optional — shown with defaults):

Global settings: max_concurrency (100), max_queue_depth (500), budget_usd_per_hour (None = off), budget_soft_margin (0.8), budget_hard_behavior ("downgrade" or "refuse"), chaos_enabled (False).

Event vocabulary

Everything observable flows through one in-process event bus (SQLite log and dashboard are subscribers):

breaker_trip · breaker_half_open · breaker_close · fallback_used (with the rung that served: cache / cheaper_model / static) · request_queued · request_shed · chaos_injected · chaos_cleared · budget_soft_ceiling · budget_hard_ceiling · manual_override

Subscribe from your own code: unsubscribe = ballast.subscribe(lambda event: ...).

Project layout

src/ballast/
  breaker.py       circuit breaker: closed/open/half-open, learned latency baseline
  backpressure.py  global concurrency gate: strict-FIFO queue + shedding
  interceptor.py   @guarded / guard() — the integration point
  fallback.py      TTL response cache + rules-based difficulty classifier
  budget.py        rolling-hour spend tracker, soft/hard ceilings
  chaos.py         time-boxed fault rules + scenario presets
  events.py        Event model + in-process pub/sub bus
  eventlog.py      SQLite audit log (attach to any runtime)
  runtime.py       configure() / status() / the global runtime
  dashboard/       FastAPI backend + live web UI + built-in demo swarm
tests/             100 tests; breaker logic is FakeClock-driven (no sleeps)
examples/
  demo.py                terminal demo: swarm → chaos → trip → fallback → recovery
  langgraph_example.py   reference integration: a LangGraph pipeline surviving an outage

Design docs in the repo root: PRD.md (product requirements), TechSpec.md (architecture), UISpec.md (dashboard screens & components).

Development

pip install -e .[dev,dashboard]
pytest                      # 100 tests, ~3s
python examples/demo.py     # terminal demo
python -m ballast.dashboard # live dashboard

Design principles worth knowing before contributing:

• All breaker timing is injectable — logic takes a Clock callable; tests drive a FakeClock and never sleep.
• Events are published outside locks — a slow or reentrant subscriber can't deadlock the call path.
• The bus swallows subscriber exceptions — observability failures must never take down a guarded call.
• Chaos rules expire on their own — an abandoned experiment can't fault a dependency forever.

Roadmap

Security posture (v1)

Ballast tracks call metadata (success/failure/latency/cost) — it never inspects or proxies request/response content. The dashboard binds to localhost with no auth; put it behind your own proxy if you expose it. Chaos injection cannot be enabled from the dashboard UI — only via environment/config.

License

MIT — see LICENSE.