agent-security-harness 4.4.0

470 security tests for AI agent systems - MCP, A2A, L402, x402 wire-protocol testing, decision governance, AIUC-1 compliance, NIST AI 800-2 aligned

Agent Security Harness

Even if an agent is properly authenticated and authorized, can it still be manipulated into unsafe or policy-violating behavior?

470 executable security tests across 32 modules. MCP + A2A + L402 + x402 wire-protocol testing. Decision-layer attack scenarios. One pip install away.

$ agent-security test mcp --url http://localhost:8080/mcp
Running MCP Protocol Security Tests v4.2...
 MCP-001: Tool List Integrity Check [PASS] (0.234s)
 MCP-002: Tool Registration via Call Injection [PASS] (0.412s)
 MCP-003: Capability Escalation via Initialize [FAIL] (0.156s)
...
Results: 8/10 passed (80% pass rate) - see report.json

Quick Start

pip install agent-security-harness

# If 'agent-security' is not found, add ~/.local/bin to your PATH:
export PATH="$HOME/.local/bin:$PATH"

# See it work immediately — no server needed:
agent-security test mcp --simulate

# Then test your real MCP server:
agent-security test mcp --url http://localhost:8080/mcp

# Test an x402 payment endpoint
agent-security test x402 --url https://your-x402-endpoint.com

See docs/QUICKSTART.md for mock server setup, rate limiting, MCP server mode, and CI/CD integration.

---

Three Layers of Agent Decision Security

Layer	What it covers	Example focus
Protocol Integrity	Prevent spoofing, replay, downgrade, diversion, and malformed protocol behavior	MCP, A2A, L402, x402 wire-level tests
Operational Governance	Validate session state, capability boundaries, platform actions, trust chains, and execution context	capability escalation, facilitator trust, provenance, session security
Decision Governance	Test whether an agent should act at all under its authority, confidence, scope, and policy constraints	autonomy scoring, scope creep, return-channel poisoning, normalization-of-deviance

---

How This Differs From Other Projects

Capability	Invariant MCP-Scan (2K stars)	Cisco MCP Scanner (865 stars)	Snyk Agent Scan (2K stars)	NVIDIA Garak (7K stars)	This framework
What it does	Scans installed MCP configs for tool poisoning	YARA + LLM-as-judge for malicious tools	Scans agent configs for MCP/skill security	LLM model vulnerability testing	Active protocol exploitation + decision governance
Approach	Static analysis	Static + LLM classification	Config scanning	Model-layer probing	Wire-protocol adversarial testing
MCP coverage	Tool descriptions, config files	Tool descriptions, YARA rules	Config files	-	18 tests: real JSON-RPC 2.0 attacks
A2A coverage	-	-	-	-	13 tests
L402/x402 coverage	-	-	-	-	85 tests
Enterprise platforms	-	-	-	-	25 cloud + 20 enterprise
APT simulation	-	-	-	-	GTG-1002 (17 tests)
Jailbreak/over-refusal	-	-	-	Yes	50 tests (25 + 25 FPR)
AIUC-1 certification	-	-	-	-	Maps to 19 of 20 testable requirements
Research backing	-	Cisco blog	-	Papers	5 DOIs + 3 NIST submissions
MCP server mode	-	-	-	-	Yes - invoke from any AI agent
Statistical testing	-	-	-	-	Wilson CIs, multi-trial
Total tests	Pattern matching	YARA rules	Config checks	Model probes	470 active tests

Use both. Scan with Invariant MCP-Scan or Cisco MCP Scanner for static analysis. Test with this framework for active exploitation. They're complementary layers.

---

Research

Five peer-reviewed preprints and three NIST submissions underpin the methodology:

Publication	DOI
Constitutional Self-Governance for Autonomous AI Agents — 12 governance mechanisms, 77 days production data, 56 agents	10.5281/zenodo.19162104
Detecting Normalization of Deviance in Multi-Agent Systems — First empirical demonstration that automated harnesses detect behavioral drift	10.5281/zenodo.19195516
Decision Load Index (DLI): A Quantitative Framework for Agent Autonomy Risk — Measuring cognitive burden of AI agent oversight	10.5281/zenodo.18217577
Normalization of Deviance in Autonomous Agent Systems — Foundational research on behavioral drift patterns	10.5281/zenodo.15105866
Cognitive Style Governance for Multi-Agent Deployments — Governance mechanisms for managing cognitive style across multi-agent systems	10.5281/zenodo.15106553

---

Related Projects

Constitutional Governance (WHY layer)

The constitutional-agent package provides the governance gates and hard constraints that complement this test harness. Six gates, 12 hard constraints, amendment protocol — enforced in code, not YAML policy files. pip install constitutional-agent.

---

Documentation

Resource	Link
Expanded Quick Start	docs/QUICKSTART.md
Full Test Inventory (470 tests)	docs/TEST-INVENTORY.md
AIUC-1 Crosswalk	docs/AIUC1-CROSSWALK.md
Advanced Capabilities	docs/ADVANCED.md
MCP Server	docs/mcp-server.md
CI/CD GitHub Action	docs/github-action.md
Payment Attack Taxonomy	docs/PAYMENT-ATTACK-TAXONOMY.md
Comparison (detailed)	docs/COMPARISON.md
Privacy & Telemetry	docs/PRIVACY.md

---

Roadmap

v3.10 -- Prove It to Auditors ✅ Shipped. v4.1 -- Compliance Evidence ✅ Shipped. v4.2 -- Incident-Tested ✅ Shipped. v4.3 -- Supply Chain + Corpus ✅ Shipped. 470 tests, 32 modules, SSP harness (8 tests), Decision Behavior Benchmark corpus (52 cases), HIDDEN_INSTRUCTION_PATTERN DRY extraction, dynamic test count, P0 bug fixes. v5.0 -- Lock the Category (H2 2026): benchmark corpus, schema standardization, longitudinal registry. Full details in ROADMAP.md.

---

Used By

Who	Use Case
FransDevelopment / Open Agent Trust Registry	OATR SDK v1.2.0 test fixtures (X4-021 through X4-030) -- Ed25519 attestation verification

Using the harness? Open a PR to add yourself, or tag us in your project.

---

Contributing

See CONTRIBUTING.md for guidelines, SECURITY_POLICY.md for security policy, and CONTRIBUTION_REVIEW_CHECKLIST.md for the PR checklist.

License

Apache License 2.0 -- see LICENSE.