agent-handshake 1.0.0b2

agent-handshake
Check that what your agent is told matches the tools it actually has

An agent's instructions and tool descriptions often name its tools, and nothing checks that those names still exist. When a tool is renamed or an MCP server adds a prefix, the model calls a tool it does not have, gets an error and often just gives up.

agent-handshake reads the request your agent sends to the model and reports where the text and the tools disagree, without calling a model.

Example

The instructions say mem_ssh; the MCP server registered the tool as mcp__mem__mem_ssh (a real case):

{
  "model": "claude-haiku-4-5",
  "max_tokens": 1024,
  "system": "Use `mem_ssh` to connect to Tower.",
  "messages": [{"role": "user", "content": "Is Tower up?"}],
  "tools": [{"name": "mcp__mem__mem_ssh", "input_schema": {"type": "object"}}]
}

$ handshake check request.json
3 rules enabled. Checked request.json (Anthropic Messages): 1 tools, 2 text segments.
✗ HS002 request.json (system): Text names `mem_ssh`; no tool has that exact name. Did you mean `mcp__mem__mem_ssh`?
1 findings (1 errors, 0 warnings).

Install

pip install --pre agent-handshake

Status: beta (1.0.0b2). --pre is needed while 1.0 is in beta: without it pip installs 0.10.0, an older and different tool (see History).

Get request.json

request.json is the request body as sent to the model, in OpenAI Chat Completions, OpenAI Responses or Anthropic Messages format. If your code calls an SDK, put the call's arguments in a dict, save it as JSON and pass the same dict to the call.

OpenAI:

import json

from openai import OpenAI

request = {
    "model": "gpt-4.1",
    "messages": [
        {"role": "system", "content": "Use `get_weather` for questions about the weather."},
        {"role": "user", "content": "Is it raining in Lisbon?"},
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                    "required": ["city"],
                },
            },
        }
    ],
}
with open("request.json", "w") as f:
    json.dump(request, f, indent=2)

response = OpenAI().chat.completions.create(**request)

Anthropic:

import json

from anthropic import Anthropic

request = {
    "model": "claude-sonnet-5-5",
    "max_tokens": 1024,
    "system": "Use `get_weather` for questions about the weather.",
    "messages": [{"role": "user", "content": "Is it raining in Lisbon?"}],
    "tools": [
        {
            "name": "get_weather",
            "input_schema": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        }
    ],
}
with open("request.json", "w") as f:
    json.dump(request, f, indent=2)

response = Anthropic().messages.create(**request)

Run

handshake check request.json

Each finding names the rule, where the text is (its path in the request) and the closest real name. Errors make the command exit with code 1; warnings do not, unless --strict is given. --config points to a handshake.yaml or pyproject.toml; without it, the one in the current directory is used.

In pytest

agent_handshake.check(body, config=None) runs the same rules on a dict and returns the findings (since 1.0.0b2).

import json

from agent_handshake import check


def test_request_matches_tools():
    with open("request.json") as fp:
        body = json.load(fp)  # or build it with your agent's own code
    assert not [f for f in check(body) if f.severity == "error"]

A complete example: examples/pytest/.

GitHub Actions

- run: |
    pip install --pre agent-handshake
    handshake check request.json

Rules

Rule pages: docs/rules/. Turning rules off, changing their severity and ignoring names: docs/configuration.md.

Known limitations

HS001 can warn about names that are not missing tools:

• Code Mode declarations. A description that declares functions as tools.x({ ... }) is read as calls to x.
• Negative forms. "Never attempt 'apply_patch' ... they don't exist" is read as naming apply_patch.
• Placeholders. "I'm going to use tool_name" is read as naming tool_name.

HS001 findings are warnings, so they do not fail the check unless --strict is used. To silence them, ignore the names or turn the rule off in handshake.yaml (or [tool.handshake] in pyproject.toml):

ignore:
  - tool_name                  # never report this name
  - {rule: HS001, ref: apply_patch}
rules:
  HS001: {enabled: false}      # or turn HS001 off

Benchmark

Measured once on the frozen configuration, on three sets that share no project: dev (used to build the rules), holdout and holdout-2 (never used to tune them). Recall is over real, fixed mismatches from the projects' history; precision is over hand-labelled findings on the current code of open-source agents. Small samples are marked by their size.

The holdout HS001 findings are Code Mode function declarations of one project (tools.x({ ... })), which HS001 reads as calls. Details and the method: benchmarks/README.md; run it with make benchmark.

Roadmap

Capturing requests from a running agent, so there is no request.json to write by hand (not started).

Contributing

See CONTRIBUTING.md.

License

Apache 2.0. See LICENSE.

History (0.x)

Releases up to 0.10.0 on PyPI are a different, closed tool: contract tests between agents. Its code is in tag contract-testing-final; see why it was closed.