toolbreaker

Why this exists

AI agents don't fail gracefully. When an upstream tool returns truncated JSON, a 429 Too Many Requests, or a 502 Bad Gateway, the agent doesn't back off — it retries. And retries. And retries. Burning tokens, burning time, and burning your budget on infinite retry loops that never change their arguments.

We call this "tool rot": the silent degradation of agent reliability when external tools fail in realistic ways. Unit tests with 200 OK responses don't catch it. Integration tests against a live MCP server don't catch it either, because they pass until they don't.

toolbreaker is a chaos-engineering harness and circuit breaker for AI agent tools. You wrap your tool functions (sync or async), inject realistic faults, and watch your agent fail — before it fails in production.

What it does

• Injects faults — truncated JSON, HTTP 429s, latency spikes, stack-trace bombs, empty {"data": null} responses, and server errors
• Detects retry loops — trips AgentRetryLoopError when an agent calls the same failing tool with the same arguments too many times
• Preserves signatures — inspect.signature(fn) still works on wrapped functions, so LangChain, MCP, CrewAI, and OpenAI function-calling schemas keep working
• Deterministic chaos — a seeded random.Random makes CI runs reproducible

Install

pip install toolbreaker

Quickstart

toolbreaker demo

This runs a zero-config simulation comparing a naive retry agent against a resilient one:

┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ Tool              ┃ Injected Fault ┃ Calls Made ┃ Loop Status ┃ Verdict     ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ Naive Retry Agent │ server_fault   │ 3          │ Caught      │ LOOP CAUGHT │
│ Resilient Agent   │ server_fault   │ 3          │ Caught      │ LOOP CAUGHT │
└───────────────────┴────────────────┴────────────┴─────────────┴─────────────┘

Pytest plugin

Add --toolbreaker-seed to any test run:

pytest --toolbreaker-seed=42

Use the toolbreaker fixture in your tests:

def test_agent_handles_truncation(toolbreaker):
    harness = toolbreaker  # RotHarness, seeded from --toolbreaker-seed (default 42)
    wrapped_tool = harness.wrap(my_mcp_tool)
    result = wrapped_tool({"query": "users"})
    assert "error" in result or result is not None

If a test fails with AgentRetryLoopError, pytest appends a reproduction line:

Reproduce with: pytest --toolbreaker-seed=42

MCP wrapper example

from toolbreaker import RotHarness, truncate_json, rate_limit

def call_tool(name: str, arguments: dict):
    # your MCP dispatch logic
    ...

harness = RotHarness(
    faults=[truncate_json(ratio=0.5), rate_limit(retry_after=30)],
    seed=42,
)
wrapped = harness.wrap_mcp(call_tool)

wrap_mcp accepts a single call_tool(name, arguments) function or a dict of {tool_name: callable}.

CLI: toolbreaker run

Inspect how your function's output mutates under every built-in fault:

toolbreaker run my_module:my_function

API

from toolbreaker import (
    RotHarness,
    CircuitBreaker,
    AgentRetryLoopError,
    truncate_json,
    rate_limit,
    payload_bomb,
    latency_spike,
    empty_lie,
    server_fault,
)

CI

GitHub Actions runs ruff check, ruff format --check, and pytest on Python 3.11, 3.12, and 3.13. Publishing to PyPI happens automatically on GitHub Release via OIDC Trusted Publishing.

License

MIT.# toolbreaker