agent-handshake 1.0.0b1

agent-handshake
Check that what your agent is told matches the tools it actually has

agent-handshake checks that what your agent is told matches the tools it actually has.

Status: beta, released (1.0.0b1). Releases up to 0.10.0 on PyPI are a different, closed tool: contract tests between agents. See why it was closed.

Install

pip install --pre agent-handshake

--pre is needed while 1.0 is in beta: without it pip installs 0.10.0, the old tool.

Check a request

handshake check request.json

request.json is a request body as sent to the model, in OpenAI Chat Completions, OpenAI Responses or Anthropic Messages format. Each finding names the rule, where the text is (its path in the request) and the closest real name. Errors make the command exit with code 1; warnings do not, unless --strict is given. Rules and settings: docs/rules/ and docs/configuration.md.

Benchmark

Measured once on the frozen configuration, on three sets that share no project: dev (used to build the rules), holdout and holdout-2 (never used to tune them). Recall is over real, fixed mismatches from the projects' history; precision is over hand-labelled findings on the current code of open-source agents. Small samples are marked by their size.

The holdout HS001 findings are Code Mode function declarations of one project (tools.x({ ... })), which HS001 reads as calls. Details and the method: benchmarks/README.md; run it with make benchmark.

Known limitations

HS001 can warn about names that are not missing tools:

• Code Mode declarations. A description that declares functions as tools.x({ ... }) is read as calls to x.
• Negative forms. "Never attempt 'apply_patch' ... they don't exist" is read as naming apply_patch.
• Placeholders. "I'm going to use tool_name" is read as naming tool_name.

HS001 findings are warnings, so they do not fail the check unless --strict is used. To silence them, ignore the names or turn the rule off in handshake.yaml (or [tool.handshake] in pyproject.toml):

ignore:
  - tool_name                  # never report this name
  - {rule: HS001, ref: apply_patch}
rules:
  HS001: {enabled: false}      # or turn HS001 off

Contributing

See CONTRIBUTING.md.

License

Apache 2.0. See LICENSE.