agent-handshake 0.7.0

agent-handshake
Contract tests for multi-agent systems

agent-handshake finds the fields that one agent reads from another agent. It tests these fields in CI. The tests use recorded outputs, so they make no LLM calls and need no API keys.

agent-handshake was called a2a-spec before version 0.7.

The problem

Agent A sends data to agent B. You change the prompt of agent A, and its output changes. For example, agent A now sends urgency instead of priority.

Agent B reads priority with a default value. It does not stop with an error. It gives a wrong result, and no test fails.

What agent-handshake does

1. It reads the code of agent B and finds the fields that agent B uses.
2. It records the output of agent A one time and saves it as JSON files in git.
3. It tests the recorded output against the fields that agent B needs. You run these tests in pytest or in CI.

Terms

Get started

You need Python 3.11 or later and a git repository.

This example uses two agents. The producer triage_agent classifies a message. The consumer resolution_agent uses that result.

# agents/triage.py
async def triage_agent(data: dict) -> dict:
    text = data["message"].lower()
    category = "billing" if "charge" in text else "general"
    return {"category": category, "priority": "high" if "urgent" in text else "low"}

# agents/resolution.py
async def resolution_agent(ticket: dict) -> dict:
    escalate = ticket.get("priority", "low") == "high"
    return {"reply": f"We are checking your {ticket['category']} issue.", "escalate": escalate}

1. Install agent-handshake.

pip install agent-handshake

2. Create the project files.

handshake init

This command creates handshake.yaml and the handshake/ directory.

3. Remove the example spec and the example scenario.

rm handshake/specs/example-spec.yaml handshake/scenarios/example-scenarios.yaml

4. Add the producer to handshake/adapters/__init__.py.

from agent_handshake import FunctionAdapter
from agents.triage import triage_agent

ADAPTERS = {
    "triage-agent": FunctionAdapter(fn=triage_agent, agent_id="triage-agent", version="1.0"),
}

5. Add inputs for the producer to handshake/scenarios/support.yaml.

scenarios:
  - name: double_charge
    input:
      message: "I was charged twice. This is urgent."
  - name: question
    input:
      message: "How do I change my address?"

6. Record the output of the producer.

handshake record

The command calls the producer one time for each scenario:

    PASS triage-agent / double_charge recorded
    PASS triage-agent / question recorded
[done] 2 snapshots saved to ./handshake/snapshots

If ADAPTERS has more than one agent, the command calls each agent with each scenario. To record one agent only, use handshake record --agent triage-agent. To give a scenario to some agents only, add agents: [triage-agent] to the scenario.

7. Make the spec from the code of the consumer.

handshake infer --producer triage-agent --consumer resolution-agent \
  --source agents/resolution.py:resolution_agent

The command writes handshake/specs/triage-agent-to-resolution-agent.yaml:

# Generated by 'handshake infer' from agents/resolution.py:resolution_agent and 2 triage-agent snapshot(s). Review before committing.
#   priority: required, read at line 2
#   category: required, read at line 3
spec:
  name: triage-agent-to-resolution-agent
  producer: triage-agent
  consumer: resolution-agent
  structural:
    type: object
    required: [priority, category]
    properties:
      priority: {type: string}
      category: {type: string}

Read the spec before you commit it. If the command shows a WARN line, read Limits.

8. Run the tests.

handshake test

PASS triage-agent-to-resolution-agent / double_charge
PASS triage-agent-to-resolution-agent / question
  2 passed  (total: 2)

9. Commit the files.

git add handshake.yaml agent_handshake agents
git commit -m "test: add agent contracts"

The committed snapshots are the baseline for the next steps.

When you change an agent

1. Change the producer. For example, change its prompt, its model or its code. In this example, the producer now sends urgency instead of priority.

2. Record the output again.

handshake record

3. Compare the new output with the committed snapshots.

handshake diff

  triage-agent / double_charge:
    FAIL priority: Field 'priority' was removed
    LOW  urgency: Field 'urgency' was added

4. Run the tests.

handshake test

FAIL triage-agent-to-resolution-agent / double_charge
    [<root>] 'priority' is a required property

5. Correct the problem.

• If the change in the producer is an error, correct the producer. Then do steps 2 to 4 again.
• If the change is necessary, change the consumer. Then run handshake infer with --force and commit the new spec and snapshots.

Do not record the snapshots again only to make a test pass. Do not delete snapshots or remove fields from a spec for this purpose.

Keep the specs up to date

A consumer can start to read a new field when you change its code. Then its spec is not complete. To find this problem, run:

handshake infer --check

FAIL triage-agent-to-resolution-agent (agents/resolution.py:resolution_agent)
    resolution_agent reads 'language' (line 3), which the spec does not cover

The command compares each spec that handshake infer made with the current code of its consumer. It reads only code, so it needs no snapshots and makes no LLM calls. The output also shows the command that makes the spec again.

Add handshake infer --check to CI. The pytest plugin also runs this check as the consumer_code test of each inferred spec.

Run the tests with pytest

agent-handshake has a pytest plugin. Add these lines to pyproject.toml:

[tool.pytest.ini_options]
handshake_config = "handshake.yaml"
testpaths = ["tests", "handshake/specs"]

pytest then runs one test for each snapshot:

$ pytest -v
handshake/specs/triage-agent-to-resolution-agent.yaml::double_charge PASSED   [ 50%]
handshake/specs/triage-agent-to-resolution-agent.yaml::question PASSED        [100%]

The usual pytest options work, for example -k, -x and --junitxml. To run the tests one time without the configuration, use pytest --handshake.

To test a live output in your own test, use the handshake_check fixture. This example is an async test, so it needs an async plugin such as pytest-asyncio.

async def test_triage_live(handshake_check):
    output = await triage_agent({"message": "I was charged twice"})
    handshake_check("triage-agent-to-resolution-agent", output)

Run the tests in CI

The tests use the committed snapshots. CI needs no API keys.

On GitHub, use the agent-handshake action. It runs handshake infer --check and handshake test, and it shows each failure as an annotation in the pull request:

# .github/workflows/contracts.yml
name: Agent contracts
on: [push, pull_request]

jobs:
  contracts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: padobrik/agent-handshake@v0.7.0

The action has these inputs: package (for example agent-handshake[semantic]), working-directory, config, check-inferred and args.

You can also run the commands yourself:

# .github/workflows/contracts.yml
name: Agent contracts
on: [push, pull_request]

jobs:
  contracts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-python@v6
        with:
          python-version: "3.12"
      - run: pip install agent-handshake
      - run: handshake infer --check
      - run: handshake test --format github

--format github shows each failure as an annotation in the pull request. For JUnit XML, use --format junit > results.xml. If your specs have semantic rules, install agent-handshake[semantic] instead.

Write a spec by hand

A spec can have three types of rules:

spec:
  name: triage-to-resolution
  version: "1.0"
  producer: triage-agent
  consumer: resolution-agent
  structural:
    type: object
    required: [category]
    properties:
      category: {type: string, enum: [billing, general]}
  semantic:
    - rule: summary_matches_message
      method: embedding_similarity
      source_field: message     # field in the producer input
      target_field: summary     # field in the producer output
      threshold: 0.5
  policy:
    - rule: no_card_numbers
      method: regex
      patterns: ['\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b']

Semantic rules need pip install "agent-handshake[semantic]". See Writing specs for all options.

Check exact values

A scenario can also give exact values for the output of an agent. handshake test and pytest compare these values with the snapshot or with the live output.

scenarios:
  - name: double_charge
    input:
      message: "I was charged twice. This is urgent."
    expectations:
      triage-agent:
        category: billing

FAIL triage-agent-to-resolution-agent / double_charge
  expectations FAIL
    expected 'category' = 'billing', got 'general'

Connect your agents

For an A2A agent, give the base URL of the agent. The adapter reads the Agent Card, finds the JSON-RPC interface and sends the scenario input. It records the agent version from the Agent Card in each snapshot.

from agent_handshake import A2AAdapter

ADAPTERS = {
    "triage-agent": A2AAdapter(url="https://triage.example.com", agent_id="triage-agent"),
}

See Writing adapters for all options.

Use with coding agents

agent-handshake contains an Agent Skill. The skill tells a coding agent how to use agent-handshake. It also tells the agent not to make a failed test pass by recording the snapshots again.

handshake skill --dir path/to/skills

This command writes path/to/skills/agent-handshake/SKILL.md. Export the skill again after you update agent-handshake.

Limits

• handshake infer reads Python consumers only. It finds data["key"], data.get("key") and "key" in data, also in nested objects (data["customer"]["tier"], data.get("customer", {}).get("tier")) and through a local name (customer = data["customer"]). It does not follow list items.
• If the consumer gives its input to another function, handshake infer cannot follow it. The command then shows a WARN line. Add the missing fields to the spec by hand.
• handshake infer --check checks only specs that contain inferred_from. Specs from agent-handshake 0.3 and earlier do not have it. Run handshake infer with --force again to add it.
• handshake diff compares with the snapshots in git. Commit the snapshots before you use it.
• A2AAdapter supports A2A 1.0 with JSON-RPC. It does not support streaming.
• Semantic rules use a local embedding model. The semantic extra installs PyTorch.

Commands

See CLI reference for all options.

Documentation

• Writing specs
• Writing adapters
• CLI reference
• CI integration
• Concepts
• Architecture
• Example project

Contributing

See CONTRIBUTING.md.

License

Apache 2.0. See LICENSE.