a2a-proof 0.14.1

Black-box contract tests for A2A agents

a2a-proof

Black-box contract tests for A2A agents.

a2a-proof discovers a deployed agent, sends real A2A requests, and checks its observable behavior. It needs no access to the agent's source code, framework, prompts, or model provider.

The official A2A TCK checks protocol conformance, A2A ITK checks interoperability between SDKs, and the A2A Inspector supports interactive debugging. a2a-proof adds repeatable contracts for the behavior of your deployed agent.

It targets A2A 1.0 over JSON-RPC, HTTP+JSON, and gRPC. The SDK compatibility layer also supports AP2 v0.2.0 agents that expose A2A 0.3 JSON-RPC, including signed mandate-to-receipt payment flows.

What you can prove

Risk	Contract
A prompt, model, or backend change alters an answer	Text, structured data, JSON Schema, and file integrity assertions
An LLM succeeds only some of the time or gets slower	Repeated trials, pass rates, parallel runs, and p50/p95 latency
A long-running task breaks after acceptance	State trajectories, stream resumption, cancellation, persistence, and push delivery
Staging no longer behaves like production	Agent Card preflight and deployment diff
Agent text leaks a secret or system prompt	Global invariants and bounded failure evidence
An agent produces an invalid payment proof	Signed AP2 mandate-chain and receipt verification

Quick start

With uv installed, try a real loopback A2A exchange without an API key or external service:

uvx a2a-proof demo
uvx a2a-proof demo --fail

The second command deliberately shows a contract failure and exits 1.

Point the runner at your agent:

uvx a2a-proof init https://agent.example.com

init reads the Agent Card and creates a2a-proof.yaml. Replace its smoke scenario with behavior your users depend on:

version: 1

agent:
  url: https://agent.example.com

defaults:
  trials: 5
  pass_rate: 0.8

scenarios:
  - name: billing dispute routing
    message: A customer says their card was charged twice for order 4815.
    latency:
      p95_seconds: 15
    expect:
      state: completed
      data:
        - path: /queue
          equals: billing-disputes
        - path: /priority
          matches: "(?i)^high$"

$ uvx a2a-proof run
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━┓
┃ Result ┃ Scenario                ┃ Trials ┃ Time ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━┩
│ PASS   │ billing dispute routing │    4/5 │ 3.2s │
└────────┴─────────────────────────┴────────┴──────┘

1 scenario passed in 3.2s

Run one scenario, save failure evidence, or compare the same contract against another deployment:

uvx a2a-proof run --scenario "billing dispute routing"
uvx a2a-proof run --format junit --output a2a-proof.xml --evidence evidence
uvx a2a-proof diff --against https://candidate-agent.example.com

GitHub Actions

After checking out the repository, one step runs its default contract as a CI check:

- uses: aspix2k/a2a-proof@v0.14.1

Set config only when the contract is not a2a-proof.yaml.

Documentation

• Writing contracts
• Assertions
• Task lifecycle
• Push notifications
• AP2 contracts
• External agent showcases
• Running in development and CI
• Configuration schema

The schema provides completion and inline validation in YAML-aware editors. init links it automatically.

Safety

Agent responses and file metadata are treated as untrusted input. Requests, response parts, regular expressions, embedded schemas, evidence, and local file access are bounded. Redirects, credential-bearing URLs, external schema references, and cross-origin interfaces are rejected by default. Remote file URLs are never fetched or written to reports.

See SECURITY.md for private vulnerability reports.

Contributions follow CONTRIBUTING.md. Licensed under the MIT License.