a2a-proof 0.7.0

Black-box contract tests for A2A agents

a2a-proof

Black-box contract tests for A2A agents.

a2a-proof discovers a deployed agent, sends real A2A requests, and checks its observable behavior. It needs no access to the agent's source code, framework, prompts, or model provider.

Why

The official A2A TCK checks protocol conformance, and the A2A Inspector supports interactive debugging. a2a-proof adds repeatable behavior and lifecycle contracts for local development and CI.

It targets A2A 1.0 over JSON-RPC, HTTP+JSON, and gRPC.

Quick start

uvx a2a-proof init https://agent.example.com
uvx a2a-proof run

init reads the Agent Card and creates a2a-proof.yaml. Add the behavior you depend on:

version: 1

agent:
  url: https://agent.example.com

scenarios:
  - name: capital of France
    message: What is the capital of France?
    expect:
      state: completed
      max_seconds: 10
      text:
        contains: Paris

$ a2a-proof run
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━┓
┃ Result ┃ Scenario          ┃ Trials ┃ Time ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━┩
│ PASS   │ capital of France │    1/1 │ 1.2s │
└────────┴───────────────────┴────────┴──────┘

1 scenario passed in 1.2s

What it covers

• Text, structured data, and file parts.
• Single-turn, multi-turn, repeated, and parallel trials.
• Agent Card, state trajectory, latency, and global leak invariants.
• Task cancellation and persistence through real A2A lifecycle operations.
• Baseline-to-candidate deployment comparison.
• Terminal, JSON, JUnit, and bounded evidence output.

Compare the same contract against a candidate deployment:

a2a-proof diff --against https://candidate-agent.example.com

The diff reports regressions, improvements, changed results, and unchanged results. It compares contract outcomes rather than raw model text; the candidate result controls the exit status.

Documentation

• Writing contracts
• Assertions
• Task lifecycle
• Running in development and CI
• Configuration schema

The schema provides completion and inline validation in YAML-aware editors. init links it automatically.

Safety

Agent responses and file metadata are treated as untrusted input. Requests, response parts, regular expressions, embedded schemas, evidence, and local file access are bounded. Redirects, credential-bearing URLs, external schema references, and cross-origin interfaces are rejected by default. Remote file URLs are never fetched or written to reports.

See SECURITY.md for private vulnerability reports.

Development

Requires Python 3.11+ and uv.

uv sync --all-groups
uv run ruff format --check .
uv run ruff check .
uv run ty check
uv run python scripts/generate_schema.py --check
uv run pytest --cov=a2a_proof
uv run mutmut run --max-children 1

The test suite includes real JSON-RPC, HTTP+JSON, and gRPC exchanges. See CONTRIBUTING.md for the contribution workflow.

License

MIT