AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Edoardo Debenedetti1, Jie Zhang1, Mislav Balunović1,2, Luca Beurer-Kellner1,2, Marc Fischer1,2, Florian Tramèr1
1ETH Zurich and 2Invariant Labs
Quickstart
pip install agentdojo
[!IMPORTANT] Note that the API of the package is still under development and might change in the future.
If you want to use the prompt injection detector, you need to install the transformers extra:
pip install "agentdojo[transformers]"
Running the benchmark
The benchmark can be run with the benchmark script. Documentation on how to use the script can be obtained with the --help flag.
For example, to run the workspace suite on the tasks 0 and 1, with gpt-4o-2024-05-13 as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:
python -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \
-ut user_task_1 --model gpt-4o-2024-05-13 \
--defense tool_filter --attack tool_knowledge
To run the above, but on all suites and tasks, run the following:
python -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \
--defense tool_filter --attack tool_knowledge
Inspect the results
To inspect the results, go to the dedicated results page of the documentation. AgentDojo results are also listed in the Invariant Benchmark Registry.Agent
Documentation of the Dojo
Take a look at our documentation.
Development set-up
Take a look at the development set-up docs.
Citing
If you use AgentDojo in your research, please consider citing our paper:
@inproceedings{
debenedetti2024agentdojo,
title={AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for {LLM} Agents},
author={Edoardo Debenedetti and Jie Zhang and Mislav Balunovic and Luca Beurer-Kellner and Marc Fischer and Florian Tram{\`e}r},
booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2024},
url={https://openreview.net/forum?id=m1YYAQjO3w}
}
1 maintainer
Metadata
1 maintainer
Release files for agentdojo 0.1.35
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| agentdojo-0.1.35.tar.gz | 720.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| agentdojo-0.1.35-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 912.8 kB
Release files / agentdojo-0.1.35.tar.gz
| Download URL | agentdojo-0.1.35.tar.gz |
|---|---|
| Size | 720.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9eacbc89d996f8656b235ad7b626bcf840b1ace7101174ca62d790c7c6d62956
|
|
BLAKE2b-256 checksum How to use checksums |
8adc2cc783d46c73297c15127d3b7971d7ba7b5cb782bf36a8b240079f7df61d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 27, 2025.
Transparency logRelease files / agentdojo-0.1.35-py3-none-any.whl
| Download URL | agentdojo-0.1.35-py3-none-any.whl |
|---|---|
| Size | 192.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
364bea4219716b716bf639f504d195943f7f6a5535d312ca41d7098704a2affd
|
|
BLAKE2b-256 checksum How to use checksums |
caa0371be72fdd08bbae0c3c5cd22e6b35390f35c9cbe189d14b72e49058bd4b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 27, 2025.
Transparency log
