cascade-scan

AI Agent security evaluation framework — automated red-teaming for LLM tool-call governance.

cascade-scan runs 8 security probes (120+ attack vectors) against a cascade-governed AI agent pipeline to evaluate its security posture. It tests injection detection, XSS, SQLi, prompt leaks, RCE, multi-step tool chains, and data exfiltration — then produces a weighted score (A+–F) and compliance-grade HTML/JSON report.

cascade-scan run
→ Injection:      18/20 blocked (90%)   ✓ PASS
→ Tool Abuse:      8/10 blocked (80%)   ✓ PASS
→ XSS:            14/16 blocked (87%)   ✓ PASS
→ SQLi:           20/20 blocked (100%)  ✓ PASS
→ Prompt Leak:    14/16 blocked (87%)   ✓ PASS
→ RCE:            18/18 blocked (100%)  ✓ PASS
→ Tool Chain:      8/8  blocked (100%)  ✓ PASS
→ Data Flow:      20/20 blocked (100%)  ✓ PASS
─────────────────────────────────────────────
Score : 92.3/100   Grade: A
Verdict: PASS

Quick Start

pip install cascade-scan

# Scan with default rules
cascade-scan run

# Add custom blocklist rules
cascade-scan run --rule name:delete_file --rule name:exec_command

# Require a minimum score (CI integration)
cascade-scan run --min-score 80 --output report.html

from cascade import DecisionPipeline
from cascade_scan import ScanEngine
from cascade_scan.probes import (
    InjectionProbe, ToolAbuseProbe, XSSProbe, SQLIProbe,
    PromptLeakProbe, RCEProbe, ToolChainProbe, DataFlowProbe,
)

pipe = DecisionPipeline(enable_injection_detection=True)

engine = ScanEngine()
engine.add_probe(InjectionProbe())
engine.add_probe(ToolAbuseProbe())
engine.add_probe(XSSProbe())
engine.add_probe(SQLIProbe())
engine.add_probe(PromptLeakProbe())
engine.add_probe(RCEProbe())
engine.add_probe(ToolChainProbe())
engine.add_probe(DataFlowProbe())

result = engine.run(pipe)
print(result.summary())
# → 8/8 probes passed, Score: 92.3/100, Verdict: PASS

CLI Reference

cascade-scan run                          # Run all probes
cascade-scan score                        # Score only
cascade-scan list-scenarios               # List built-in attack scenarios
cascade-scan run --probes xss,rce         # Run specific probes
cascade-scan run --rule name:delete_file  # Add blocklist rule
cascade-scan run --min-score 80           # Set pass threshold
cascade-scan run --fail-below 80          # Exit 1 if score < 80 (CI)
cascade-scan run --output report.html     # Save HTML report
cascade-scan run --output report.json     # Save JSON report
cascade-scan evolve --iterations 5        # Iterative evaluation
cascade-scan baseline save baseline.json  # Save current as baseline
cascade-scan baseline compare baseline.json  # Compare vs baseline
cascade-scan import-scenario custom.json  # Import custom attack scenarios

Security Probes

Attack Scenarios

Pre-built scenarios test end-to-end threat models:

Scoring

Scores are computed as a weighted average of probe pass rates:

Score = Σ(weight × pass_rate) / Σ(weight) × 100

--min-score defaults to 70. Set higher for stricter requirements.

Reports

HTML reports are self-contained (inline CSS, zero JavaScript) — suitable for compliance archives and team sharing. JSON reports are structured for CI tooling.

cascade-scan run --output security-report.html    # open in any browser
cascade-scan run --output ci-report.json           # parse in CI pipeline

Architecture

cascade-scan
├── src/cascade_scan/
│   ├── __init__.py          # Public API
│   ├── engine.py            # ScanEngine — probe orchestration
│   ├── scorer.py            # SecurityScorer — weighted A+–F scoring
│   ├── report.py            # HTML/JSON report export
│   ├── evolve.py            # Evolver — iterative evaluation
│   ├── baseline.py          # BaselineManager — save/load/compare
│   ├── cli.py               # Command-line interface
│   ├── probes/
│   │   ├── __init__.py      # Probe base class + ProbeResult
│   │   ├── injection.py     # 20+ injection patterns
│   │   ├── tool_abuse.py    # 10 dangerous tool types
│   │   ├── xss.py           # 16 XSS vectors
│   │   ├── sqli.py          # 20 SQL injection vectors
│   │   ├── prompt_leak.py   # 16 prompt leak vectors
│   │   ├── rce.py           # 18 RCE vectors
│   │   ├── tool_chain.py    # 8 multi-step attack chains
│   │   └── data_flow.py     # 20 exfiltration vectors
│   ├── scenarios/
│   │   ├── __init__.py
│   │   └── registry.py      # 5 built-in attack scenarios
│   └── _models.py           # Shared data models
├── tests/                   # 78 tests
│   ├── test_engine.py
│   ├── test_probes.py
│   ├── test_scorer.py
│   ├── test_report.py
│   ├── test_scenarios.py
│   ├── test_xss.py
│   ├── test_sqli.py
│   ├── test_prompt_leak.py
│   ├── test_rce.py
│   ├── test_tool_chain.py
│   └── test_data_flow.py
├── pyproject.toml
├── README.md
└── LICENSE

Built on cascade (C₁ gate, C₃ selector, C₄ feedback, injection detection, SHA-256 audit chain).

License

MIT