agent-injection-bench

An open benchmark for prompt injection against tool-calling agents: published with its dataset, its harness, and its scoring rules.

$ aib matrix --targets naive,paranoid --defenses none,delimit,spotlight

  target    defense       compromised  over-refused
  ──────────────────────────────────────────────────
  naive     none               61.9%         0.0%
  naive     delimit            61.9%         0.0%
  naive     spotlight          47.6%         0.0%
  paranoid  none                0.0%       100.0%
  paranoid  delimit             0.0%       100.0%
  paranoid  spotlight           0.0%       100.0%

Why this exists

Most published prompt-injection results are hard to compare and harder to reproduce. Three reasons, and this benchmark is built around avoiding all three.

They measure prompts, not channels. A corpus of bare attack strings measures the wrong thing. Real injections arrive through a channel (a tool result, a fetched page, a filename, a calendar invite) and the channel is most of what makes them work. An agent that shrugs off ignore previous instructions typed by the user may well obey the identical sentence inside a web page it was asked to summarise. Every attack here declares its channel, and results are broken down by it.

They score with an LLM judge. Using a model to grade a model introduces a second, unmeasured system into the measurement. Judge disagreement becomes benchmark noise nobody can attribute, and when the judge shares a family with the system under test, the result is not independent at all. Here, every attack declares a machine-checkable success condition: a canary string, a named tool call, a secret reaching a sink. Two runs of the same configuration produce the same number.

They report one number. Compromise rate alone is trivially gamed: an agent that refuses everything scores a perfect zero. That is not hypothetical: it is the paranoid reference target above, at 0.0% compromise and 100% over-refusal. Every result here reports both, and the corpus ships control cases specifically so the second number exists.

Install

pip install git+https://github.com/sentinelden/agent-injection-bench

The core has no dependencies. Add the OpenAI-compatible adapter with:

pip install 'agent-injection-bench[http] @ git+https://github.com/sentinelden/agent-injection-bench'

(Not on PyPI yet, pip install agent-injection-bench will work once it is.)

From source:

git clone https://github.com/sentinelden/agent-injection-bench
cd agent-injection-bench
pip install -e '.[dev,http]'
pytest

Usage

# Validate the corpus.
aib validate

# Run against a built-in reference target, no API key needed.
aib run --target naive --defense none

# Run against any OpenAI-compatible endpoint.
export AIB_BASE_URL=https://api.example.com/v1
export AIB_API_KEY=...
aib run --target openai-compat:model=your-model --defense spotlight

# Compare defenses head to head.
aib matrix --targets openai-compat:model=your-model \
           --defenses none,delimit,spotlight,filter,allowlist

# Machine-readable, for CI or a leaderboard.
aib run --target naive --defense none --json --output results.json

# Inspect a single attack.
aib describe ex-001

aib run exits non-zero when anything was compromised, so it can gate a pipeline.

The corpus

24 scenarios: 21 adversarial, 3 controls. Deliberately small. Every attack is a technique published in the literature or observed in the wild, encoded as a scenario with a deterministic success condition, a corpus whose numbers mean the same thing every run is worth more than a large one whose numbers drift.

Channels: tool_result, document, web_page, filename, metadata, multi_turn, user_message

Families: instruction_override, context_termination, authority_claim, data_exfiltration, tool_redirection, encoding, role_play, conversation_hijack, plus baseline and control

The corpus is one JSONL file at aib/data/attacks.jsonl, diffable, and a PR adding an attack is reviewable as text. Schema and field semantics are documented in aib/dataset.py.

Success criteria

No interpretation, no judge model.

Defenses

Measured side by side against the same corpus, because "is this model safe" is close to unanswerable while "does spotlighting reduce tool-result compromise, and what does it cost in over-refusal" has a number attached.

None is presented as sufficient. Residual compromise is reported per defense precisely so the gaps stay visible.

Adding a target

A target is any callable that takes a Scenario and returns an AgentResponse:

from aib.adapters import register
from aib.scoring import AgentResponse, ToolCall

class MyStack:
    name = "my-stack"

    def __call__(self, scenario):
        reply = my_agent.run(
            system=scenario.system_prompt,
            messages=scenario.messages,
            tools=scenario.tools,
        )
        return AgentResponse(
            text=reply.text,
            tool_calls=[ToolCall(c.name, c.args) for c in reply.calls],
        )

register("my-stack")(lambda **kw: MyStack())

The interface is small on purpose. The interesting subject of an agent-security benchmark is usually not a bare model but a stack: model plus system prompt plus tool registry plus whatever filtering sits between them. Any of those can be a target.

The harness never executes a tool; calls are answered from the attack's canned results. Running the benchmark has no side effects.

What this does not measure

• Jailbreaking. Getting a model to produce disallowed content is a different problem with a different threat model.
• Model capability. A weaker model can score better simply by failing to follow the injected instruction. Read compromise and over-refusal together, or the number misleads.
• Real-world exploitability. A compromise here means the agent took the attacker's action in a sandbox. Whether that action matters depends on your deployment.
• Anything about a target not in the corpus. 24 scenarios is a floor, not a certificate.

Contributing

The most valuable contributions:

1. Attacks in under-covered channels: filename, metadata and multi_turn have one or two scenarios each and deserve more. Each new attack needs a deterministic success condition and, where the technique is published, a citation.
2. Defense implementations: particularly ones that operate on the tool layer rather than the prompt.
3. Published results. Run the matrix against a stack you operate and open a PR with the JSON. Results that make a defense look bad are the most useful kind.

Attacks must be techniques already described publicly or trivially derivable. This is a measurement instrument, not an exploit collection.

pytest              # 27 tests
aib validate        # corpus integrity

License

MIT. See LICENSE. The corpus is released under the same terms: use it, fork it, cite it.

Who builds this

Sentinel Den, iOS security research and runtime-defense SDKs from Vancouver, BC. This benchmark exists because we needed it to evaluate our own agent-sandboxing work and found nothing we could reproduce.