Skip to main content

agent-failure-lab

A catalogue of the ways LLM agents fail in production. Each failure is reproduced deterministically without an API key, and each comes with a tested fix you can import as a guard.

CI PyPI Python versions License: MIT

pip install agent-failure-lab
aflab run duplicate-side-effect
$ aflab run duplicate-side-effect
duplicate-side-effect  (side-effects)
A retried tool call charges the customer twice
Scripted model, no network. The result is the same on every run.

Broken variant: FAILURE OBSERVED
  charges         2
  tool_calls      2
  replayed_calls  0
  Order A-1001 was charged 2 times (98.00 EUR in total).

Fixed variant: no failure
  charges         1
  tool_calls      2
  replayed_calls  1
  The retry reused the stored result, so the card was charged once.

The broken variant fails and the fixed variant does not.

The catalogue

Twelve failures so far. Each links to a write-up of what happens, why, how to spot it in your own system, the fix, and what the fix costs.

Failure Category What goes wrong Fix
duplicate-side-effect side-effects A retried tool call charges the customer twice Idempotency keys
infinite-tool-loop control-flow The model repeats the same call without progress and burns tokens Loop detection and a step budget
tool-output-injection security Instructions hidden in a tool result hijack the next action Quarantine untrusted output and gate sensitive tools
hallucinated-tool-arguments tool-use The model invents an enum value and the tool stores it Schema validation with bounded repair
silent-context-truncation context-and-cost History is cut without any error and the model loses a constraint A context budget that pins instructions and fails loudly
runaway-cost context-and-cost With no spending ceiling, a run keeps paying for more steps A cost and token ceiling with a hard stop
resume-replays-completed-steps side-effects After a crash, the restarted agent repeats work that already finished A step journal for safe resume
tool-timeout-hang control-flow A tool never returns and the whole run stalls waiting for it Deadlines with cancellation and partial results
parallel-tool-race side-effects Two parallel tool calls overwrite each other's update Serialise calls that change the same resource
ambiguous-tool-selection tool-use Two tools with overlapping descriptions get confused Description linting and clearer descriptions
false-success-claim tool-use The agent reports success after a tool returned an error Postcondition checks before an answer is accepted
memory-poisoning security Untrusted content saved to memory is later followed as an instruction Provenance tags and a filter on memory writes

The same list, with the guards each fix uses and which failures can also run against a real model, is in docs/catalog.md. docs/taxonomy.md groups the failures by where they come from.

Why this exists

Most write-ups about agent failures describe them in prose. You read that an agent "retried a payment and charged twice", agree it sounds bad, and have no way to check whether your own loop would do the same. The fixes are usually a paragraph of advice.

This project turns each failure into something you can run. A scripted model plays the part of an agent that misbehaves in one specific way, and a fake world records what actually happened: how many charges, which emails went out, how many tokens were spent. The same scenario runs twice, once without protection and once with the fix, and the difference in the world is the result. The fixes are ordinary classes you can use in your own loop.

Using it

List the failures:

aflab list

Run one, with both variants and a verdict:

aflab run tool-output-injection
aflab run tool-output-injection --trace     # include the step by step trace
aflab compare tool-output-injection         # side by side metrics and traces

Run all of them, which is also what CI does:

aflab run --all --fail-on-unfixed

--fail-on-unfixed exits with status 1 only when a fixed variant still shows its failure. Output can be switched to --format json or --format md, and aflab report writes a Markdown or JSON summary of every failure to a file with --output.

Exit status 2 means the command was wrong (an unknown failure name, options that do not go together). The message is one line, not a traceback.

Using the guards in your own loop

The guards do not depend on the simulator. Each is a small object with up to five hooks that your loop calls: before_model, after_model, before_tool, after_tool and on_step_end. A hook returns None to leave things alone, a replacement value, Block(result) to skip a tool call, or Stop(reason) to end the run.

from agent_failure_lab.guards import LoopDetector, StepBudget

guards = [LoopDetector(max_repeats=3), StepBudget(25)]

Three runnable examples show how to wire them up:

docs/guards.md documents every guard, and docs/porting.md explains how to move one into a framework.

Running against a real model

Four failures can run against a live model, because they are judged by what happened in the fake world and not by how a scripted model was written: duplicate-side-effect, tool-output-injection, hallucinated-tool-arguments and parallel-tool-race.

pip install "agent-failure-lab[live]"
export OPENAI_API_KEY=...        # on Windows PowerShell: $env:OPENAI_API_KEY = "..."
aflab run duplicate-side-effect --live --model gpt-4o-mini

A live run is labelled as such, calls the API and costs money, and its result is not deterministic. The output says what happened and does not claim that the fix works or fails for that model. Live mode has not been checked against the real OpenAI API by the author, so treat it as unverified until you have run it yourself. The tests exercise it against a fake client only.

Adding a failure

aflab new my-new-failure --category tool-use

This creates the package, a README skeleton and a test, and registers the failure. Then it is your turn to describe the failure properly. docs/writing-a-failure.md walks through it, and CONTRIBUTING.md covers the checks. Ideas that are not built yet are in docs/ideas.md.

What this is, and what it is not

The scenarios use a scripted model that misbehaves on purpose. They show a failure mechanism and the effect of a fix. They say nothing about how often a real model fails, or whether yours would.

  • It is not a benchmark of model reliability. No figure here compares models.
  • It is not an agent framework. It has a small loop so the scenarios can run, and the guards are meant to be used in whatever loop you already have.
  • It is not a hosted service.
  • No guard makes an agent safe. Each closes one specific gap, and each write-up says what it leaves open. The prompt injection quarantine, for example, only labels untrusted text. The capability gate is the control you can rely on.
  • Token and cost figures use a rough estimate of four characters per token and illustrative prices. They show proportions, not real bills.
  • It has been tested on Linux. The CI runs the tests on Windows as well, but the author has not used the tool on Windows by hand.

Roadmap

Not promises, only the direction. Possible new failures (schema drift in tool responses, retry storms, stale retrieval, truncated structured output) are listed in docs/ideas.md. Live mode for more of the failures, and live adapters for providers other than OpenAI, would need a way to judge the result that does not depend on how the model phrases things.

Contributing

Bug reports, new failure ideas and fixes are welcome. Start with CONTRIBUTING.md. Please report security problems privately, as described in SECURITY.md.

License

MIT. See LICENSE.

Author

Hasnat Ahmad (GitHub, LinkedIn).

Metadata

Release files for agent-failure-lab 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agent-failure-lab 0.1.0
File Size Uploaded Provenance Actions
agent_failure_lab-0.1.0.tar.gz 117.5 kB With provenance Details for agent_failure_lab-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agent-failure-lab 0.1.0
File Interpreter ABI Platform Actions
agent_failure_lab-0.1.0-py3-none-any.whl Python 3 none any Details for agent_failure_lab-0.1.0-py3-none-any.whl

Total release size: 222.3 kB

Release files / agent_failure_lab-0.1.0.tar.gz

Download URL agent_failure_lab-0.1.0.tar.gz
Size 117.5 kB
Tags Source
SHA-256 checksum
How to use checksums
4bd9f5e7967135f8ec6ea80425e26ade6c3219eaf9b057b8d3dbef50f19024bb
BLAKE2b-256 checksum
How to use checksums
a562fdf1a75c2747e0b0933c42306b7f570a114e9639f95d6b9ad3d34f829f2a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.

Transparency log

Release files / agent_failure_lab-0.1.0-py3-none-any.whl

Download URL agent_failure_lab-0.1.0-py3-none-any.whl
Size 104.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
848ec713f124057969c0775054e2401e4f1998d84f0fde31be36d9bc83481c01
BLAKE2b-256 checksum
How to use checksums
9b3593793404063f9d92476645640a5414fa70532d102f45867d6a2676c4675c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.

Transparency log

Security for agent-failure-lab 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for agent_failure_lab-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 2, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page