rogue-llm

Adversarial LLM proxy for testing structural vs behavioral agent security.

Every existing agent red-teaming tool sends adversarial prompts to test your agent. rogue-llm does something no other tool does: it replaces the model itself with a hostile one that actively tries to exfiltrate your sensitive data through 7 different attack strategies.

$ rogue-llm serve --secrets secrets.json

╔══════════════════════════════════════════════════════════════╗
║                    rogue-llm v0.3.0                          ║
║            Adversarial LLM for Structural Security           ║
╚══════════════════════════════════════════════════════════════╝

  Hostile server on http://127.0.0.1:8642/v1
  Loaded 4 secrets to exfiltrate

  ✗ LEAK 'SUP-8842' via verbatim
  ✗ LEAK '27315' via base64+verbatim
  ✗ LEAK '71.8' via verbatim

╔══════════════════════════════════════════════════════════════╗
║  VERDICT: BEHAVIORAL SECURITY ONLY                           ║
║  Your agent leaked 3/4 secrets when the model was hostile    ║
╚══════════════════════════════════════════════════════════════╝

Why this matters

If your agent leaks data through a rogue model → your security is behavioral (depends on model cooperation). If it blocks the leak → your security is structural (the architecture prevents exfiltration regardless).

Structural security is the only security that survives prompt injection, model compromise, and future attacks you haven't imagined yet.

Install

pip install rogue-llm

Quick start — Security Scan (new in v0.3.0)

Run all 7 attack profiles against your agent and get a security score:

from roguellm import SecurityScan, ALL_PROFILES

scan = SecurityScan(
    secrets=["cost_27315", "margin_71.8", "SUP-8842"],
    profiles=ALL_PROFILES,
)

results = scan.run(my_agent_function)

print(results.summary())
# Security Score: 85/100 (B)
# Overall Verdict: BEHAVIORAL
# Profiles: 6/7 passed

results.save_html("security-report.html")  # Interactive dashboard
results.save_json("security-report.json")  # Machine-readable

Your agent_function receives a base_url and should run your agent against it:

def my_agent_function(base_url: str):
    client = openai.OpenAI(base_url=base_url, api_key="not-needed")
    # ... run your agent normally ...

MCP Server Adversarial Testing (new in v0.3.0)

No other tool does this. Test whether your MCP server's tool design leaks sensitive data when the model is hostile:

from roguellm import MCPScan, MCPTool

tools = [
    MCPTool(
        name="get_customer",
        description="Get customer by ID",
        parameters={"id": {"type": "string"}},
        handler=lambda args: {"name": "Acme", "revenue": "2.3M", "notes": "VIP"},
        sensitive=True,
    ),
    MCPTool(
        name="send_email",
        description="Send email",
        parameters={"to": {"type": "string"}, "body": {"type": "string"}},
        handler=lambda args: {"status": "sent"},
    ),
]

scan = MCPScan(tools=tools, secrets=["2.3M", "VIP"])
results = scan.run()
print(results.summary())
# MCP Security Score: 42/100 (F)
# Overall Verdict: BEHAVIORAL

Or from the CLI with a JSON tool definition file:

rogue-llm mcp-audit --tools tools.json --secrets secrets.json --min-score 80

Where tools.json defines your MCP tools with mock results:

[
  {
    "name": "get_customer",
    "description": "Get customer by ID",
    "parameters": {"id": {"type": "string"}},
    "mock_result": {"name": "Acme", "revenue": "2.3M", "notes": "VIP"},
    "sensitive": true
  },
  {
    "name": "send_email",
    "description": "Send email",
    "parameters": {"to": {"type": "string"}, "body": {"type": "string"}},
    "mock_result": {"status": "sent"}
  }
]

The tool_filter parameter simulates MCP access control — block specific tools to test whether your permissions model is sufficient:

scan = MCPScan(
    tools=tools,
    secrets=["2.3M"],
    tool_filter=lambda name, args: name != "send_email",  # Block email
)

Six ways to test your agent

Option 1: One-line CLI scan (new in v0.3.0)

The fastest path to a security score — one command, no code:

rogue-llm scan-agent python my_agent.py --secrets secrets.json

This starts the hostile server, sets OPENAI_BASE_URL for your agent, runs all 7 attack profiles, and prints a scored report:

  ▶ Running full_attack...
  ▶ Running data_exfil...
  ▶ Running encoding_evasion...
  ...

  ╔══════════════════════════════════════════════════════════╗
  ║  Security Score:  85/100  Grade: B                       ║
  ║  Verdict: BEHAVIORAL                                     ║
  ╚══════════════════════════════════════════════════════════╝

  ✓ full_attack            STRUCTURAL    leaked 0/4
  ✓ data_exfil             STRUCTURAL    leaked 0/4
  ✗ encoding_evasion       BEHAVIORAL    leaked 2/4
  ...

Use it in CI with a minimum score gate:

# Fail the build if score < 80
rogue-llm scan-agent python my_agent.py \
  --secrets secrets.json \
  --min-score 80 \
  --html report.html \
  --output report.json

Options:

• --profiles full_attack,stealth — run specific profiles only
• --min-score 80 — exit code 1 if score is below this (CI gate)
• --html report.html — save the visual HTML dashboard
• --output report.json — save machine-readable JSON

Option 2: Security Scan API (comprehensive)

The scan runs every attack profile against your agent and produces a 0–100 security score with a letter grade:

from roguellm import SecurityScan, AttackProfile

scan = SecurityScan(
    secrets=["cost_27315", "margin_71.8", "SUP-8842"],
    profiles=[
        AttackProfile.FULL_ATTACK,
        AttackProfile.DATA_EXFIL,
        AttackProfile.EMAIL_ONLY,
        AttackProfile.ENCODING_EVASION,
        AttackProfile.TOOL_ABUSE,
        AttackProfile.STEALTH,
        AttackProfile.SYSTEM_PROMPT,
    ],
)

results = scan.run(my_agent)

assert results.score == 100, results.summary()  # Perfect score = structural
assert results.grade == "A+"
assert results.verdict == "STRUCTURAL"

Option 3: pytest fixture

The rogue_server fixture is available automatically — no imports needed:

@pytest.mark.rogue_secrets(["cost_27315", "margin_71.8", "SUP-8842"])
def test_agent_structural_security(rogue_server):
    """My agent must not leak internal data, even to a hostile model."""
    reply = my_agent.run(base_url=rogue_server.base_url)
    # On teardown, rogue_server asserts STRUCTURAL (no leaks detected)

Option 4: Python API

from roguellm import RogueTest, AttackProfile

# Test with a specific attack profile
test = RogueTest(
    secrets=["cost_27315", "margin_71.8"],
    profile=AttackProfile.STEALTH,  # or any of the 7 profiles
)
with test.start() as server:
    my_agent.run(base_url=server.base_url)

assert test.verdict == "STRUCTURAL", test.report()

Option 5: MCP server audit

See MCP Server Adversarial Testing above.

Option 6: CLI (serve + point)

# Terminal 1: Start the hostile server
rogue-llm serve --secrets secrets.json --port 8642

# Terminal 2: Point your agent at it
OPENAI_BASE_URL=http://localhost:8642/v1 python my_agent.py

7 Attack profiles

Each profile tests a different real-world threat vector:

from roguellm import AttackProfile, ALL_PROFILES

# Run a specific profile
test = RogueTest(secrets=["X"], profile=AttackProfile.STEALTH)

# Iterate all profiles
for profile in ALL_PROFILES:
    test = RogueTest(secrets=["X"], profile=profile)
    with test.start() as server:
        my_agent(server.base_url)
    print(f"{profile.name}: {test.verdict}")

SSE Streaming support

The server fully supports OpenAI-compatible SSE streaming — critical because most production agents use stream=True:

import openai

client = openai.OpenAI(
    base_url="http://127.0.0.1:8642/v1",
    api_key="not-needed",
)

# Works exactly like real OpenAI streaming
for chunk in client.chat.completions.create(
    model="rogue-hostile-1",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
):
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Both streaming and non-streaming responses include tool calls with the same hostile payload — your agent's streaming code path is tested against the same attacks.

HTML Security Report

The security scan generates a self-contained HTML dashboard:

results = scan.run(my_agent)
results.save_html("report.html")

The report includes:

• Score circle with color-coded grade (green/yellow/orange/red)
• Per-profile cards showing pass/fail, leak count, and progress bar
• Leak details for every detected secret
• Raw JSON data in an expandable section
• Dark theme, self-contained — no external dependencies

Framework wrappers

Zero-config functions for common frameworks:

from roguellm.frameworks import test_agent, scan_agent, test_openai

# Generic: any callable that takes base_url
result = test_agent(my_agent_fn, secrets=["X"])

# Multi-profile scan
results = scan_agent(my_agent_fn, secrets=["X"])
print(results.grade)  # "A+"

# OpenAI SDK: provides a pre-configured client
result = test_openai(my_openai_agent, secrets=["X"])

Security Benchmark + Badge (new in v0.3.0)

Run a standardized security benchmark across common agent patterns and get an embeddable badge:

rogue-llm benchmark --badge badge.svg --output benchmark.json

  ▶ No Tools (baseline) — Agent with no tools — only text replies...
  ▶ Read-Only Tools — Agent with tools that read internal data...
  ▶ Read + Email Tools — Agent with read tools AND an email tool...
  ▶ Streaming Output — Agent using SSE streaming...

Security Benchmark: 42/100 (F)
Overall Verdict: BEHAVIORAL
Scenarios: 4

  ✗ No Tools (baseline)           28/100 ( F)  BEHAVIORAL
  ✗ Read-Only Tools               42/100 ( F)  BEHAVIORAL
  ✗ Read + Email Tools            14/100 ( F)  BEHAVIORAL
  ✗ Streaming Output              85/100 ( B)  BEHAVIORAL

The badge is an SVG file you can embed in your README:

![security](https://raw.githubusercontent.com/your-org/your-repo/main/badge.svg)

Python API:

from roguellm.benchmark import SecurityBenchmark

bench = SecurityBenchmark()
results = bench.run()
print(results.summary())

results.save_badge("badge.svg", label="my-agent security")
results.save_json("benchmark.json")

assert results.overall_score >= 80, "Security score too low"

Diff mode — cross-framework parser differential testing

No other tool does this. Send the same malformed tool-call response to multiple agent frameworks and compare what each one actually executes:

pip install rogue-llm[all]
rogue-llm diff
rogue-llm diff --json -o report.json
rogue-llm diff --category duplicate-keys

What it tests

17 crafted malformed responses across 7 categories:

Adapters

Exit code 1 if security-relevant divergences are found — plug it into CI.

Scan mode

Already have a transcript? Scan it for leaked secrets:

rogue-llm scan transcript.txt --secrets secrets.json

The detector finds secrets in plaintext, base64, hex, rot13, reversed text, spaced digits, and nested encodings (up to 3 levels deep).

GitHub Action

Add structural security testing to your CI pipeline:

# .github/workflows/security.yml
name: Agent Security

on: [push, pull_request]

jobs:
  rogue-llm:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: samuelrojas-dev/rogue-llm@v1
        with:
          frameworks: "openai langchain-core litellm"

The action installs rogue-llm, runs differential testing across the specified frameworks, and fails the build if security-relevant divergences are found.

Action inputs

Action outputs

Endpoints

Examples

The examples/ directory has runnable demos:

pytest examples/test_with_pytest.py -v
python examples/test_with_api.py

How is this different from...

• hackagent, probeagent, Giskard: Send adversarial prompts to a normal model. Test behavioral security only.
• LlamaFirewall, Pipelock: Add guardrails around the model. Don't test whether the architecture itself is secure.
• agent-audit-kit: Scans MCP server configurations. Static analysis, not adversarial testing.
• rogue-llm: Replaces the model with a hostile one. Tests whether your architecture prevents leaks, regardless of model behavior. 7 attack profiles, MCP server adversarial testing, streaming support, security scoring, HTML reports.

Related research

This tool implements the CompromisedLLM methodology from agent-security-lab, which demonstrated that only architectural (structural) defenses survive a fully compromised model.

License

MIT