Agent Security Harness

Even if an agent is properly authenticated and authorized, can it still be manipulated into unsafe or policy-violating behavior?

603 executable security tests across 44 modules (verified 2026-08-02 via scripts/count_tests.py). MCP + A2A + L402 + x402 wire-protocol testing, plus UCP/ACP merchant-journey, AP2 mandate-chain, Fireblocks x402 hardening, Visa TAP / Mastercard Agentic Token funding-instrument, and denial-of-settlement finality conformance across the full agentic-payments stack. Decision-layer attack scenarios. One pip install away.

$ agent-security test mcp --url http://localhost:8080/mcp
Running MCP Protocol Security Tests v4.15.0...
 MCP-001: Tool List Integrity Check [PASS] (0.234s)
 MCP-002: Tool Registration via Call Injection [PASS] (0.412s)
 MCP-003: Capability Escalation via Initialize [FAIL] (0.156s)
...
Results: 8/10 passed (80% pass rate) - see report.json

Illustrative output. A target the harness cannot reach, or that answers without servicing the request, reports INCONCLUSIVE — never PASS. See v4.13.1 for why that distinction is enforced rather than assumed.

If this evidence discipline is useful in your agent-security work, star this repository to follow releases.

Evidence before coverage

Every claim in this project is bounded by the E1-E5 Evidence Class Taxonomy: observation, runtime characterization, enforcement, persistence/replay resistance, and isolation. A second axis, I0-I2, states who produced the oracle: self-authored, independently reimplemented, or an independent sensor the target does not control. Strength and independence are different properties, so both are cited. A result is not promoted beyond what its retained artifact and execution record demonstrate. Author-performed mappings and test runs are not independent certification.

The AIUC-1 Evidence Field Guide is a plain-language companion that applies this same taxonomy. It adds one distinction the ladder above does not encode: whether evidence is mapped (a documented requirement relationship), executed (a recorded run against a stated target and pinned revision), or independently reviewed (assessed by a qualified outside party). Those describe the status of evidence and are orthogonal to E1-E5, which describes its strength. A mapping alone is E1-level material regardless of how many requirements it covers. The taxonomy in this repository is canonical; the field guide is hosted outside it and is not version-pinned.

OWASP Agentic AI v1.1 Threat Coverage Report — commit-pinned mapping from the full T1–T17 taxonomy to executable tests: 13 direct, 4 partial, 0 not evidenced, across 96 mapped tests and 66 named OWASP scenarios. Mitigation-control validation is tracked separately from threat coverage (11 validated, 10 partial, 1 guidance-only), and every gap, evidence class and reproduction command is in the report. (T1–T15 submission view · canonical mapping)

Adapted from OWASP Agentic AI — Threats and Mitigations v1.1 under CC BY-SA 4.0. A test-capability report — not a certification, conformance claim, or OWASP endorsement. The adjudication is author-performed and is not independent review.

Quick Start

pipx install agent-security-harness

pipx is the recommended install because this is a command-line tool. It builds an isolated environment and puts agent-security on your PATH, with no sudo and no PATH edits. Get it with brew install pipx, apt install pipx, or python3 -m pip install --user pipx.

python3 -m venv .venv && source .venv/bin/activate
pip install agent-security-harness

error: externally-managed-environment

pipx upgrade agent-security-harness

# Confirm which build you're on:
agent-security --version

# See it work immediately — no server needed:
agent-security test mcp --simulate

# Then test your real MCP server:
agent-security test mcp --url http://localhost:8080/mcp

# Test an x402 payment endpoint
agent-security test x402 --url https://your-x402-endpoint.com

# Human-oversight surface: reviewer exposure (T10) and agent-to-human
# manipulation (T15). An unreachable target reports INCONCLUSIVE, never PASS.
agent-security test hitl --url http://localhost:8080

# Which tests back a given OWASP threat, scenario or mitigation control
python scripts/owasp_agentic_select.py --threat T16
python scripts/owasp_agentic_select.py --control P5-REA-001

See docs/QUICKSTART.md for mock server setup, rate limiting, MCP server mode, and CI/CD integration.

Three Layers of Agent Decision Security

How This Differs From Other Projects

Star counts verified 2026-08-02 via the GitHub API. Invariant Labs' mcp-scan now redirects to snyk/agent-scan and is listed once rather than as two separate projects. Competitor rows describe what each tool does by design; a dash means the capability is outside its stated scope, not a defect.

What the table does not measure. Every row above is coverage: what gets tested, and how much. Coverage says nothing about what a result licenses you to claim. That is what the E1-E5 Evidence Class Taxonomy is for, and it is applied to this project's own output first: the AIUC-1 crosswalk is E1 material regardless of covering 19 of 20 requirements, and this harness holds no I2 evidence at all, because it reads protocol responses the target itself emits. No claim here is deliberately made about how other projects handle this.

One external check. The Research backing row says self-authored because it is. The one exception is #304, where an outside party reimplemented the receipt-claim oracle from the published contract and replayed a pinned corpus. One reproduction, of one corpus, by one party, is not independent review of the harness as a whole, and this project still does not have that.

Use both. Scan with Snyk Agent Scan or Cisco MCP Scanner for static analysis. Test with this framework for active exploitation. They're complementary layers.

Research

Seven public preprints and notes deposited on Zenodo (not represented as peer-reviewed publications) and three NIST submissions underpin the methodology. Every DOI below was re-verified on 2026-08-02 by content negotiation against doi.org — title and authorship confirmed:

On citation counts. These records carry internal citation lineage across later work in this portfolio. An OpenAlex cited_by audit on 2026-08-02 found 30 citation edges and 0 qualifying independent citations — every edge is a self-citation, collapsing to nine duplicate/version records across three title families. Nothing here should be read as third-party validation or scholarly adoption. The one external check this project has received is an independent reproduction, listed under Used By.

Correction (2026-08-02). Two entries previously in this table cited DOIs belonging to other researchers — 10.5281/zenodo.15105866 (a MALDI mass-spectrometry dataset by Ranes et al.) and 10.5281/zenodo.15106553 (an e-learning article by Toshtemirov). They were attributed here to "Normalization of Deviance in Autonomous Agent Systems" and "Cognitive Style Governance for Multi-Agent Deployments". No Zenodo record under those titles by this author was located, so both rows were removed rather than re-pointed. The surviving Detecting Normalization of Deviance in Multi-Agent Systems record (19195516) is unaffected and was already listed separately.

Related Projects

Constitutional Governance (WHY layer)

The constitutional-agent package provides the governance gates and hard constraints that complement this test harness. Six gates, 12 hard constraints, amendment protocol — enforced in code, not YAML policy files. pip install constitutional-agent.

Documentation

Roadmap

Current: v4.13.1 (2026-08-02). Recent shipped work — v4.5 skill supply chain and governance modification · v4.6–v4.9 payment-stack depth (AP2 mandate chain, UCP/ACP merchant journey, card-network agentic tokens, settlement finality, Fireblocks x402) · v4.10 benchmark integrity · v4.11–v4.12 decision-governance corpus currency and provenance repair · v4.13 OWASP Agentic v1.1 T1–T17 coverage mapping and the human-in-the-loop harness · v4.13.1 a correctness fix to that harness (see CHANGELOG.md).

Next — Standards & Evidence. Reproducible settlement-time payment evidence, a methodology paper, and the attestation/evidence schema submitted to a standards venue. Coverage breadth and test count are explicitly not goals. Full detail and the anti-goals in ROADMAP.md.

Used By

Independent reproduction

The only external check this project has received. @VrtxOmega wrote a separate Node verifier for the portable receipt-claim oracle fixtures and ran it against a pinned commit and fixture hash, matching all 11 RCL results including both acceptance controls — so it did not pass by rejecting everything.

The exchange also produced two corrections to this repository: signature_algorithm now names the actual encoding and states plainly that it is not RFC 8785 JCS, and coverage_gaps now declares that freshness is exercised only in the stale direction. Both are in fixtures/rcl/.

It is a reproduction of a pinned artifact by one external party, submitted as a report and explicitly not as a contribution, endorsement, certification, or adoption. It is not a substitute for independent review of the harness as a whole, which this project still does not have.

Using the harness? Open a PR to add yourself, or tag us in your project.

Contributing

See CONTRIBUTING.md for guidelines, SECURITY_POLICY.md for security policy, and CONTRIBUTION_REVIEW_CHECKLIST.md for the PR checklist.

Citation

If you cite this work in research:

Saleme, M. K. (2026). Agent Security Harness — multi-protocol agent security testing framework. ORCID: 0009-0003-6736-1900. https://github.com/msaleme/red-team-blue-team-agent-fabric

Related Zenodo preprints are listed in full under Research; that table is the single source of truth for DOIs in this repository. Do not cite a DOI for this work that does not appear there.

License

Apache License 2.0 -- see LICENSE.