doomarena-taubench 0.0.4

TauBench extensions for DoomArena

TauBench experiments

This repository contains tools and scripts for defining and evaluating threat models in the TauBench framework. TauBench focuses on LLM agents in tool-augmented environments, providing a way to simulate realistic adversarial attacks in domains like retail and airline customer service.

Overview

The framework provides a structured way to:

• Simulate tool-based adversarial scenarios against LLM agents
• Measure metrics like Attack Success Rate (ASR), Task Success Rate (TSR), and stealthiness of attacks
• Compare defenses or model variants across structured multi-turn tasks
• Evaluate robustness of tool-using agents in realistic settings

Domains

TauBench currently supports:

• Retail: Multi-tool shopping agents that handle product searches, returns, and recommendations
• Airline: Agents for booking flights, managing itineraries, and accessing sensitive account information

Installation

1. Install this package

# install main from this repo
pip install -e doomarena/taubench

# or install from pypi
pip install doomarena-taubench

2. Install taubench

git clone https://github.com/sierra-research/tau-bench scripts/tau-bench
pip install -e git+https://github.com/sierra-research/tau-bench.git#egg=tau_bench

3. You may also need to set your OpenRouter API key:

export OPENROUTER_API_KEY=<your-api-key>
export OPENAI_API_KEY=<your-api-key>

Usage

Example (Retail malicious user attack):

cd doomarena/taubench/src/doomarena/taubench
python scripts/attack_script.py \
  --config scripts/malicious_user_retail_attack.yaml

Experiment Configuration Options

• combined_retail_attack.yaml
  Runs multiple retail attack types in a single config for comprehensive evaluation.

• malicious_catalog_fixed_injection_retail_attack.yaml
  Inserts a malicious product entry into the retail catalog with a fixed injection strategy.

• malicious_catalog_retail_attack.yaml
  Injects a dynamic malicious catalog item to mislead the retail agent.

• malicious_user_airline_attack.yaml
  Simulates a malicious user attempting to manipulate the airline booking assistant.

• malicious_user_fixed_injection_airline_attack.yaml
  Similar to the above but with fixed injection content for consistent attack setup.

• malicious_user_retail_attack.yaml
  Tests how a retail agent handles adversarial inputs from a user aiming to bypass rules or gain unauthorized benefits.

Each config specifies:

• Attack type and injection method
• Success filters
• Prompt construction (system + few-shot examples)

Results and Metrics

Experiment results are stored in the results/taubench directory, organized by the datetime when they were created. Each results folder includes:

• Metadata about the attack configuration, agent, and dataset used
• CSV files containing metrics such as:
  • Attack Success Rate (ASR)
  • Task Success Rate (TSR)
  • Attack Stealth Rate
  • Tool call counts and usage breakdowns
  • Input/output token counts
  • Step-by-step interaction logs with the agent

You can analyze per-task outcomes to understand failure modes, effectiveness of the attacks, and behavior of tool-augmented agents under adversarial pressure.

Project Structure

├── README.md 
├── pyproject.toml 
├── src/doomarena/taubench/                  
    ├── attack_gateway.py            # Entry point for attack orchestration
    ├── data/                        # JSON datasets for different domains
    │   ├── airline_classification.json
    │   ├── retail_classification.json
    │   └── sample_airline.json
    ├── filters/                     # Filters for selecting relevant agent actions
    │   ├── is_get_product_details_action_filter.py
    │   └── is_respond_action_filter.py
    ├── success_filters/            # Criteria for judging if attack succeeded
    │   ├── airline_info_leak_success_filter.py
    │   ├── llm_judge.py
    │   ├── retail_refund_success_filter.py
    │   ├── retail_secrets_success_filter.py
    │   └── send_certificate_success_filters.py
    ├── system_prompt_config/       # Prompt configurations and few-shot data
    │   ├── system_prompt_initialization.py
    │   ├── utils.py
    │   ├── dan_mode/
    │   │   ├── dan_mode_airline.txt
    │   │   ├── dan_mode_retail.txt
    │   │   └── dan_mode_retaildb.txt
    │   ├── few_shot_examples/
    │   │   ├── airline_few_shot.json
    │   │   ├── retail_few_shot.json
    │   │   └── retaildb_few_shot.json
    │   └── tools/
    │       ├── airline_tools.json
    │       └── retail_tools.json
    ├── scripts/                    # YAML attack configs and the main runner
    │   ├── combined_retail_attack.yaml
    │   ├── malicious_catalog_fixed_injection_retail_attack.yaml
    │   ├── malicious_catalog_retail_attack.yaml
    │   ├── malicious_user_airline_attack.yaml
    │   ├── malicious_user_fixed_injection_airline_attack.yaml
    │   ├── malicious_user_retail_attack.yaml
    │   └── attack_script.py
├── tests/
    ├── test_data
        ├── taubench_config.yaml
    ├── __init__.py
    ├── test_run_tau_bench_attack.py
    ├── test_taubench_attack_config.py
    ├── test_taubench_attack_gateway.py

Contributing

Contributions are welcome! You can extend this framework by:

1. Adding new attack vectors -- new prompt injections or misuse of tools in the airline or retail domains
2. Testing additional agent models -- evaluate how different LLMs or fine-tuned agents perform under attack
3. Implementing new evaluation metrics -- define novel task-specific or stealth-aware success criteria