agent-injection-bench

An open benchmark for prompt injection against tool-calling agents: published with its dataset, its harness, and its scoring rules.

$ aib matrix --targets naive,paranoid --defenses none,delimit,spotlight

  target    defense       compromised  over-refused
  ──────────────────────────────────────────────────
  naive     none               81.8%         0.0%
  naive     delimit            81.8%         0.0%
  naive     spotlight          39.4%         0.0%
  paranoid  none                0.0%       100.0%
  paranoid  delimit             0.0%       100.0%
  paranoid  spotlight           0.0%       100.0%

Why this exists

Most published prompt-injection results are hard to compare and harder to reproduce. Three reasons, and this benchmark is built around avoiding all three.

They measure prompts, not channels. A corpus of bare attack strings measures the wrong thing. Real injections arrive through a channel (a tool result, a fetched page, a filename, a calendar invite) and the channel is most of what makes them work. An agent that shrugs off ignore previous instructions typed by the user may well obey the identical sentence inside a web page it was asked to summarize. Every attack here declares its channel, and results are broken down by it.

They score with an LLM judge. Using a model to grade a model introduces a second, unmeasured system into the measurement. Judge disagreement becomes benchmark noise nobody can attribute, and when the judge shares a family with the system under test, the result is not independent at all. Here, every attack declares a machine-checkable success condition: a canary string, a named tool call, a secret reaching a sink. Two runs of the same configuration produce the same number.

They report one number. Compromise rate alone is trivially gamed: an agent that refuses everything scores a perfect zero. That is not hypothetical: it is the paranoid reference target above, at 0.0% compromise and 100% over-refusal. Every result here reports both, and the corpus ships control cases specifically so the second number exists.

Install

pip install agent-injection-bench          # core, no dependencies
pip install 'agent-injection-bench[http]'  # adds the OpenAI-compatible adapter

From source:

git clone https://github.com/sentinelden/agent-injection-bench
cd agent-injection-bench
pip install -e '.[dev,http]'
pytest

Usage

# Validate the corpus.
aib validate

# Run against a built-in reference target, no API key needed.
aib run --target naive --defense none

# Run against any OpenAI-compatible endpoint.
export AIB_BASE_URL=https://api.example.com/v1
export AIB_API_KEY=...
aib run --target openai-compat:model=your-model --defense spotlight

# Compare defenses head to head.
aib matrix --targets openai-compat:model=your-model \
           --defenses none,delimit,spotlight,filter,allowlist

# Machine-readable, for CI or a leaderboard.
aib run --target naive --defense none --json --output results.json

# Inspect a single attack.
aib describe ex-001

aib run exits non-zero when anything was compromised, so it can gate a pipeline.

The corpus

38 scenarios: 33 adversarial, 5 controls. Deliberately small. Every attack is a technique published in the literature or observed in the wild, encoded as a scenario with a deterministic success condition, a corpus whose numbers mean the same thing every run is worth more than a large one whose numbers drift.

Channels: tool_result, document, web_page, filename, metadata, multi_turn, user_message, tool_chain

Families: instruction_override, context_termination, authority_claim, data_exfiltration, tool_redirection, encoding, role_play, conversation_hijack, tool_chain, plus baseline and control

The corpus is one JSONL file at aib/data/attacks.jsonl, diffable, and a PR adding an attack is reviewable as text. Schema and field semantics are documented in aib/dataset.py.

Chains

Some attacks only pay off across several steps, so a single call measures them badly. A tool_chain scenario can have steps: what arrives after the first turn, in order. The success condition is checked against everything the agent did across all of them.

"steps": [
  {"channel": "memory", "from_call": "save_memory", "task": "Pay the electricity bill.", "new_session": true}
]

• content is what arrives, or from_call takes it from what the agent actually passed to that tool. That is the intermediate state a multi-step compromise depends on. If the agent never made the call, the chain has broken there and the later steps do not run.
• new_session: true starts a fresh conversation, so nothing survives except what the step carries. Memory poisoning and agent hand-offs work this way.
• A chain with no steps still needs several tool calls in one session, answered from the canned results.

The chains cover the lethal trifecta, cross-tool request forgery, poisoned and shadowed tool descriptions, memory written in one session and acted on in another, delayed tool invocation, agent hand-offs, self-replicating messages, and data carried out through a search query, a Markdown image or a redirect on an allowed host.

agenticguard/ has a companion corpus that asks the next question: given that a model was compromised, what does a policy layer let through? It replays the calls the model would make, with the decision expected at each one.

Success criteria

No interpretation, no judge model.

A target answers tool calls from the scenario's canned results and keeps going, up to max_rounds (default 5) on the OpenAI-compatible adapter.

Defenses

Measured side by side against the same corpus, because "is this model safe" is close to unanswerable while "does spotlighting reduce tool-result compromise, and what does it cost in over-refusal" has a number attached.

None is presented as sufficient. Residual compromise is reported per defense precisely so the gaps stay visible.

Adding a target

A target is any callable that takes a Scenario and returns an AgentResponse:

from aib.adapters import register
from aib.scoring import AgentResponse, ToolCall

class MyStack:
    name = "my-stack"

    def __call__(self, scenario):
        reply = my_agent.run(
            system=scenario.system_prompt,
            messages=scenario.messages,
            tools=scenario.tools,
        )
        return AgentResponse(
            text=reply.text,
            tool_calls=[ToolCall(c.name, c.args) for c in reply.calls],
        )

register("my-stack")(lambda **kw: MyStack())

The interface is small on purpose. The interesting subject of an agent-security benchmark is usually not a bare model but a stack: model plus system prompt plus tool registry plus whatever filtering sits between them. Any of those can be a target.

The harness never executes a tool; calls are answered from the attack's canned results. Running the benchmark has no side effects.

What this does not measure

• Jailbreaking. Getting a model to produce disallowed content is a different problem with a different threat model.
• Model capability. A weaker model can score better simply by failing to follow the injected instruction. Read compromise and over-refusal together, or the number misleads.
• Real-world exploitability. A compromise here means the agent took the attacker's action in a sandbox. Whether that action matters depends on your deployment.
• Anything about a target not in the corpus. 38 scenarios is a floor, not a certificate.

Contributing

The most valuable contributions:

1. Attacks in under-covered channels: filename, metadata and multi_turn have one or two scenarios each and deserve more, and chains can always use more stages. Each new attack needs a deterministic success condition and, where the technique is published, a citation.
2. Defense implementations: particularly ones that operate on the tool layer rather than the prompt.
3. Published results. Run the matrix against a stack you operate and open a PR with the JSON. Results that make a defense look bad are the most useful kind.

Attacks must be techniques already described publicly or trivially derivable. This is a measurement instrument, not an exploit collection.

pytest              # 44 tests
aib validate        # corpus integrity

License

MIT. See LICENSE. The corpus is released under the same terms: use it, fork it, cite it.

Changes in 0.2.0

• Chains: 12 multi-step scenarios and 2 controls, with steps in the format.
• The OpenAI-compatible adapter now delivers tool results. It sent the first request and stopped, so a model never read what a tool returned, and every attack that arrives in a tool result could not land. Results from 0.1.0 understate that channel. Re-run them.
• The naive target passes everything it read into a tool call, not the first 120 characters, and reads tool descriptions. Two older attacks now land on it, so its baseline on the original 21 went from 61.9% to 71.4%.

Who builds this

Sentinel Den, iOS security research and runtime-defense SDKs from Vancouver, BC. This benchmark exists because we needed it to evaluate our own agent-sandboxing work and found nothing we could reproduce.