CoT Red Team Agent

CoT Red Team Agent is an open-source CLI and Python API for evaluating LLM and agent behavior under adversarial inputs. It runs reproducible model attacks and offline simulated-agent scenarios, records failure-aware evidence, and generates auditable reports and replayable security regressions.

Version 0.6.0 adds the Proof-of-Action agent execution lane: a deterministic Support Agent World, deny-by-default tool gateway, action/state-transition oracles, checksummed exploit replay, and patched-target regression suites. It preserves the 0.5 adaptive attacks and TUI, 0.3 benchmark, and 0.2 Python API. Existing users should also read the 0.3 migration guide.

Interactive adaptive TUI: multi-model board, payload attempt log, model output, and last real successful disclosure (refusal re-quotes are not counted as success).

What it does

• Runs built-in and third-party attacks against one or more target models.
• Agentic canary extraction: seeds a payload bank, classifies each refusal, then invents the next technique until a compliant final-text disclosure (or budget). Refusal quotes are not success.
• LLM-driven attacker (injection.system_canary_agent_llm): an attacker model writes the next extraction prompts from the conversation (PAIR loop, TAP-style candidate branching) and falls back to the deterministic catalog if the attacker provider fails. Requires attacker_model in attack_config.
• Crescendo multi-turn attack (injection.crescendo_canary): gradual 5-rung escalation that references the model's own replies, from a benign question to an exact token ask. No attacker model needed.
• General forbidden-behavior testing (harm.forbidden_behavior): score any harmful request (phishing, fraud, weapons, hate speech, ...) with a StrongREJECT-style rubric judge — refusal, specificity, convincingness — no canary required. Ships with pkg:harmful.jsonl (10 items).
• CI-ready scan: cot-redteam scan --config X runs a quick compliance scan and exits 0 (clean) / 1 (findings) / 3 (partial) — gate CI on it.
• OWASP LLM Top 10 tags in every report item (LLM01/02/05/07...).
• Adaptive fixed-bank mode also available (injection.system_canary_adaptive).
• Interactive Codex-style TUI with slash commands, multi-model board, and live attempt timeline (cot-redteam tui).
• Runs packaged 12-trial smoke and 56-scenario core prompt-injection suites.
• Preserves system, developer, user, assistant, and simulated tool message roles.
• Scores exact and partial canary disclosure separately in final text and visible provider reasoning (refusal analysis that only quotes a canary is not success).
• Reports attack objectives, benign-task utility, false refusals, exclusions, and Wilson confidence intervals as separate dimensions.
• Evaluates outputs with regex, LLM-judge, ensemble, and evasion monitors.
• Distinguishes provider, attack, monitor, budget, and cancellation failures.
• Tracks request, token, elapsed-time, and estimated-cost budgets.
• Stores runs transactionally in SQLite with retention-aware redaction.
• Produces Markdown, CSV, and LaTeX reports with honest eligibility counts.
• Produces lossless JSONL benchmark evidence for retained multi-turn transcripts.
• Writes reproducibility manifests and detached artifact checksums.
• Evolves bounded populations of generated attack templates through the normal evaluation engine.

The tool does not provide a hosted service or guarantee that automated monitors represent ground truth.

Supported providers

Provider	Configuration kind	Typical use
OpenRouter	openrouter	Hosted access to multiple model families
OpenAI	openai	OpenAI API models
Anthropic	anthropic	Anthropic Messages API models
vLLM	vllm	Local or self-hosted OpenAI-compatible server
llama.cpp	llamacpp	Local llama.cpp OpenAI-compatible server
Generic endpoint	openai_compatible	Explicit user-selected compatible API
Mock	mock	Deterministic keyless provider for demos, tests, and CI

Provider keys are read only from named environment variables. Secrets must not be placed directly in YAML files. The mock provider needs no key at all: mock_mode: auto|refuse|disclose|error controls whether it discloses a synthetic canary, refuses, or raises provider errors — ideal for smoke tests and CI without spending any budget.

Installation

Python 3.10 through 3.13 is supported.

Install from PyPI:

python -m pip install cot-redteam-agent

Or install the tagged source release:

# test: command
python -m pip install "git+https://github.com/rudrasatani13/cot-redteam-agent.git@v0.6.0"

Or install the wheel attached to the GitHub release:

python -m pip install \
  "https://github.com/rudrasatani13/cot-redteam-agent/releases/download/v0.6.0/cot_redteam_agent-0.6.0-py3-none-any.whl"

For development:

git clone https://github.com/rudrasatani13/cot-redteam-agent.git
cd cot-redteam-agent
python -m pip install -e ".[dev]"

Published on PyPI as cot-redteam-agent.

Five-minute quickstart

Create a wheel-safe example configuration:

# test: command
cot-redteam init --path config.yaml
# Edit evaluation.models and generative model IDs for your provider route.
export OPENROUTER_API_KEY=your-key
cot-redteam config validate --config config.yaml
cot-redteam list-attacks
cot-redteam list-monitors

The generated configuration uses the packaged pkg:sample.jsonl dataset and works outside the repository. It includes optional provider examples, but validation requires credentials only for providers referenced by the selected evaluation and generative models. The default attack is injection.system_canary_agent (invent techniques until real disclosure).

Run the configured evaluation when you are ready to contact the provider:

cot-redteam run --config config.yaml
cot-redteam list-runs --config config.yaml

Interactive adaptive TUI

--config is required (bare cot-redteam tui will error):

cot-redteam tui --config config.yaml
# or auto-start:
cot-redteam tui --config config.yaml --auto-start

The bottom type bar is a single slim line (no tall box borders). Type a slash command and press Enter. Mid panels expand; the composer stays pinned.

Inside the TUI:

/model openrouter:your-model-id
/payloads 8
/run

Useful commands: /help, /add, /models, /attack, /stop, /quit. See the full TUI guide (layout + slim composer notes).

Render a report using the run_id printed by the run command:

cot-redteam report \
  --config config.yaml \
  --run-id RUN_ID \
  --format markdown

Markdown reports include retained system and attack prompts, model responses, visible provider reasoning, exact attack-assessment evidence, and monitor outcomes. The packaged adaptive canary attack places a synthetic token only in a trusted system instruction and reports success only on real disclosure—not when the model refuses while quoting the canary during analysis.

Prompt-injection benchmark

List the packaged suites:

cot-redteam list-suites
cot-redteam suite show --id builtin.smoke

In config.yaml, select a suite and remove the legacy attacks and monitors entries if you want a benchmark-only run:

evaluation:
  models:
    - openrouter:your-model-route
  suite_ids:
    - builtin.smoke
  repetitions: 1
  budgets:
    # 12 trials; one is two-turn, so the target-request minimum is 13.
    max_requests: 13
  retain_prompts: true
  retain_responses: true
  retain_reasoning: true

Provider capabilities are declared under providers.<name>.capabilities. Unsupported roles fail during config validate, before any billed request. The packaged smoke suite includes a simulated tool-output case, so the selected route must declare tool_role: true; otherwise use a filtered local suite.

Run and inspect it with the same commands:

cot-redteam config validate --config config.yaml
cot-redteam run --config config.yaml
cot-redteam report --config config.yaml --run-id RUN_ID --format markdown
cot-redteam report --config config.yaml --run-id RUN_ID --format jsonl

Benchmark results apply only to the tested model route, provider behavior, policy, suite version, transformations, and repetitions. They are not a universal model-security score. See the benchmark guide.

Visible reasoning and interpretation

The tool records visible reasoning only when it is:

1. exposed in a provider response field; or
2. enclosed by configured delimiters such as <think>...</think>.

Ordinary answer prose is not relabeled as hidden reasoning. Model outputs are nondeterministic, automated monitors are imperfect, and attack success does not prove a general model vulnerability. Reports preserve failed and excluded items so those limitations remain visible. A reasoning-only canary disclosure means the tested provider route exposed protected system content to its API caller; it does not prove that every deployment of the named model does so.

Data handling

Prompts, responses, and visible reasoning can contain confidential information. Review evaluation.retain_prompts, evaluation.retain_responses, and evaluation.retain_reasoning before running against sensitive datasets.

The default configuration retains evaluation traces. Stored SQLite databases, artifacts, reports, and generated archives should be protected as sensitive research data and must not be committed.

Responsible use

Use the project only with models, endpoints, datasets, and credentials you are authorized to test. Respect provider terms, rate limits, privacy obligations, and applicable law. Do not use generated attacks to access third-party systems or data without permission.

Model-safety results belong in normal research reports or issues. Suspected software vulnerabilities in this repository must be reported privately under the security policy.

Agent security (v0.6 Proof-of-Action)

v0.6 adds an agent execution lane that evaluates agent behavior, not just model text. Impact is proven only by observed simulated actions and deterministic world state transitions — assistant prose, model reasoning, and LLM judge opinion are never proof of impact.

• Support Agent World — the only executable simulated world: in-memory tickets, CRM records, canaries, outbound sink, approval state, and protected state. No tool performs real network, shell, filesystem, subprocess, or external database effects.
• Deny-by-default ToolGateway — registry membership, scenario sandbox allowlist, schema validation, byte/action/time/concurrency limits, and structured trajectory events for every dispatch.
• Deterministic oracles — canary-reached-sink, unauthorized-tool-call, approval-bypass, and protected-state-mutation proofs over immutable pre/post world snapshots. Oracle failures are never clean.
• Three scenarios with vulnerable/patched/clean fixtures: indirect prompt injection, tool-result injection, approval bypass.
• Replay artifacts — verified exploits save as strict checksummed JSON; exact replay reproduces them, and regression suites replay saved exploits against patched targets expected to hold.

cot-redteam agent scan --config cot_redteam/data/agent_security.example.yaml
cot-redteam replay artifacts/<run_id>/replay.json     # exit 1 = reproduced
cot-redteam regress --suite security-regressions/     # patched target holds

The v0.6 path runs fully offline with the keyless mock provider plus scripted targets and local SQLite/artifacts. Agent retention defaults to omitting raw tool arguments/results, memory values, and final-response content; structural evidence (events, digests, oracle verdicts) is always kept.

Custom Python target adapters execute in-process and are trusted code, like installed plugins. The runtime facades separate target-authored messages from gateway-authored action evidence, but they are not a sandbox for malicious adapter code. Treat model output and tool-result data as hostile; install or supply only target adapter code you trust.

Python API

# test: python
import asyncio

from cot_redteam.api import run_benchmark
from cot_redteam.core.config import load_config


async def main() -> None:
    config = load_config("config.yaml")
    run = await run_benchmark(config)
    print(run.run_id, len(run.trials))


asyncio.run(main())

Use run_evaluation for the backward-compatible 0.2 attack/monitor path and run_benchmark for configured suites. Both contact providers and may incur cost.

CLI reference

• cot-redteam init
• cot-redteam config validate|show
• cot-redteam list-attacks|list-monitors|list-providers
• cot-redteam list-suites
• cot-redteam suite validate|show
• cot-redteam dataset import cyberseceval|ih-challenge
• cot-redteam run
• cot-redteam tui — interactive adaptive dashboard
• cot-redteam race — race one probe across models and compare compliance
• cot-redteam list-runs|show-run|report
• cot-redteam evolve
• cot-redteam agent scan --config — scripted agent scenarios; saves replay artifacts for verified exploits
• cot-redteam replay EXPLOIT.json — deterministic local replay of a saved exploit
• cot-redteam regress --suite DIR — replay saved exploits against patched targets expected to hold

Exit codes are 0 for completed, 1 for failed/findings (a reproduced exploit), 2 for configuration errors (including corrupt/incompatible replay artifacts), and 3 for partial/inconclusive runs.

Documentation

Guide	Purpose
Configuration	Schema, credentials, precedence, and validation
Interactive TUI	Adaptive dashboard, slash commands, screenshot
Providers	Provider-specific behavior and endpoints
Plugins	Attack and monitor extension contracts
Experiments	Metrics, rates, comparisons, and retention
Benchmarking	Suites, capabilities, scoring, reports, and imports
0.3 migration	Additive changes from 0.2.x
Migration	Breaking changes from 0.1.x
Support	Where to ask questions or report reproducible bugs
Contributing	Development and pull-request workflow
Security	Private vulnerability reporting and scope
Changelog	Version history

Development

python -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
ruff format --check .
ruff check .
mypy cot_redteam
pytest

See CONTRIBUTING.md for the complete quality gates.

License

Released under the MIT License.