AgentSpec

pytest-style contracts for what an LLM agent does: which tools it calls, in what order, how often, with which arguments, and behind which approvals. Checked deterministically, offline, with zero dependencies.

spec = (ContractSet("refund_agent")
        .must_precede("check_refund_policy", "process_refund")   # policy gates the refund
        .must_not_call("delete_account")                          # never
        .must_call_at_most("web_search", n=5))                    # cost cap

spec.check(session).assert_all_pass()       # in CI, on a recorded trace
FastGuard(spec).enforce("delete_account")   # or live, before the tool runs -> raises Blocked

The problem

Most agent evals grade the final answer. But agents do their damage in the steps: a refund issued without the policy check, a delete_account the model decided was "cleanup", 47 paid searches for one question. The final answer can look perfect while the trajectory is wrong, and an LLM judge that only sees the answer can't tell.

These are rules, not opinions, so they should be checked like unit tests: same trace, same verdict, every time, without paying for a model call. That's what AgentSpec does.

What it does

• Contracts on tool calls — must_call, must_not_call, must_call_before, must_call_at_most / at_least / exactly, must_call_in_sequence, within_steps, never_after, eventually, must_call_with_arg.
• Argument-aware rules — where= predicates: path_not_under("/workspace"), host_not_in([...]), arg_gt("amount", 500), regex, combinators.
• Approval gates — requires_approval("deploy", correlate="env"): an approval for prod does not unlock staging.
• Severities and a decision — each rule is BLOCK / APPROVAL / WARN / AUDIT; a report rolls up to allow, block or require_approval, and points at the offending call.
• Live enforcement — check a call before it runs. FastGuard keeps incremental state, so a check costs the same at call 10 and call 10,000.
• Trace adapters — Anthropic Messages, OpenAI Chat Completions + Responses, OpenTelemetry GenAI / OpenInference spans, or plain dicts. No provider SDK needed.
• pytest native — assert_all_pass(), plus fixtures registered as a pytest plugin.

A real example

examples/why_agentspec.py is a refund agent in OpenAI tool-calling format. Its final answer is polite and plausible; its steps are not. Actual output:

Final answer the customer saw:
  All done! I've processed your $240 refund for order 5582. It will appear in 3-5 business days. Sorry about the broken item!

Tool calls in the trace:
  lookup_order -> process_refund -> delete_account

[1] Audit of the finished trace

  AgentSpec Report — 'refund_agent'
  --------------------------------------------------
  [✓] must_call(lookup_order): 'lookup_order' was called
  [✗] must_precede(check_refund_policy, process_refund) @call[1]: 'process_refund' was called before any 'check_refund_policy'  → fix: call 'check_refund_policy' before 'process_refund'
  [✗] must_not_call(delete_account) @call[2]: 'delete_account' was called 1 time(s) but must never be called  → fix: remove the 'delete_account' call or take the tool away from the agent
  [✓] must_not_call(process_refund) [conditional]: no 'process_refund' call matched the condition
  --------------------------------------------------
  2 passed, 2 failed, 0 warned
  Overall: FAIL  ·  Decision: block

[2] Same calls through a live guard, checked BEFORE each one runs
  lookup_order     {'order_id': '5582'} -> allow
  process_refund   {'order_id': '5582', 'amount': 240} -> block
  delete_account   {'order_id': '5582'} -> block

Quickstart

pip install llm-agentspec

1. Write contracts and check a session

from agentspec import ContractSet, AgentSession, ToolCall

spec = (ContractSet("research_agent")
        .must_call("search")                         # did the research
        .must_call_before("search", "summarize")     # ...before summarizing
        .must_not_call("delete_file")                # never deletes files
        .must_call_at_most("search", n=3))           # cost cap

# What your agent actually did (normally built by an adapter, see below)
session = AgentSession(tool_calls=[
    ToolCall(name="search", args={"query": "agent testing"}),
    ToolCall(name="summarize"),
    ToolCall(name="delete_file", args={"path": "/tmp/cache"}),
])

report = spec.check(session)
print(report)
print(report.decision.value)

  AgentSpec Report — 'research_agent'
  --------------------------------------------------
  [✓] must_call(search): 'search' was called
  [✓] must_call_before(search, summarize): 'search' (step 0) correctly precedes 'summarize' (step 1)
  [✗] must_not_call(delete_file) @call[2]: 'delete_file' was called 1 time(s) but must never be called  → fix: remove the 'delete_file' call or take the tool away from the agent
  [✓] must_call_at_most(search, n=3): 'search' called 1 time(s) (limit: 3)
  --------------------------------------------------
  3 passed, 1 failed, 0 warned
  Overall: FAIL  ·  Decision: block

block

2. Put it in your pytest suite

# test_my_agent.py
from agentspec import ContractSet, AgentSession, ToolCall

SPEC = (ContractSet("research_agent")
        .must_call("search")
        .must_call_before("search", "summarize")
        .must_not_call("delete_file"))


def run_my_agent(task: str) -> AgentSession:
    # stand-in for your agent: return the tool calls it made
    return AgentSession(tool_calls=[ToolCall(name="summarize"),
                                    ToolCall(name="delete_file")])


def test_research_agent_follows_the_rules():
    SPEC.check(run_my_agent("Summarize recent AI news")).assert_all_pass()

$ pytest test_my_agent.py -q
    def test_research_agent_follows_the_rules():
>       SPEC.check(run_my_agent("Summarize recent AI news")).assert_all_pass()
E       AssertionError: AgentSpec 'research_agent' failed 3 contract(s):
E         - must_call(search): 'search' was never called (calls made: ['summarize', 'delete_file'])
E         - must_call_before(search, summarize): 'search' was never called (required before 'summarize')
E         - must_not_call(delete_file) @call[1]: 'delete_file' was called 1 time(s) but must never be called

test_my_agent.py:17: AssertionError
FAILED test_my_agent.py::test_research_agent_follows_the_rules - AssertionErr...
1 failed in 0.01s

3. Enforce rules live, before a tool runs

from agentspec import ContractSet, FastGuard, matchers as m

spec = (ContractSet("coding_agent")
        .must_not_call("write_file", where=m.path_not_under("/workspace"))
        .must_call_at_most("web_search", n=20, per_seconds=60)
        .requires_approval("run_shell", correlate="cmd"))

guard = FastGuard(spec)

def check(name, args):
    print(f"{name}({args}) -> {guard.check_call(name, args).decision.value}")

check("write_file", {"path": "/workspace/app.py"})
check("write_file", {"path": "/workspace/../etc/passwd"})
check("run_shell", {"cmd": "pytest"})
guard.approve(cmd="pytest")                 # a human said yes to *this* command
check("run_shell", {"cmd": "pytest"})
check("run_shell", {"cmd": "rm -rf /"})     # the approval doesn't carry over

write_file({'path': '/workspace/app.py'}) -> allow
write_file({'path': '/workspace/../etc/passwd'}) -> block
run_shell({'cmd': 'pytest'}) -> require_approval
run_shell({'cmd': 'pytest'}) -> allow
run_shell({'cmd': 'rm -rf /'}) -> require_approval

In your tool dispatcher: guard.enforce(name, args) (raises Blocked or ApprovalRequired), run the tool, then guard.commit(name, args, result=...).

Getting your agent's trace in

Adapters work on live SDK objects or plain JSON you saved, so the same contracts run in CI against fixtures with no API key:

from agentspec.adapters import (
    from_anthropic_messages,   # Anthropic Messages: tool_use, server_tool_use, mcp_tool_use blocks
    from_openai_messages,      # OpenAI Chat Completions tool_calls, Responses API function_call items
    from_otel_spans,           # OTel GenAI execute_tool spans, OpenInference TOOL spans
    from_dicts,                # anything else: [{"name": ..., "args": {...}}, ...]
)

session = from_anthropic_messages(messages)   # the list you send back to messages.create
spec.check(session).assert_all_pass()

examples/case_study_research_agent/ runs six contracts over two Anthropic-format conversations: a good run passes, a runaway run (five searches, then delete_file) fails with the offending call indexes.

Contract reference

severity= is accepted by must_call, must_not_call, must_precede, must_call_at_most, never_after, within_steps and eventually; requires_approval always uses APPROVAL and the rest are BLOCK. Ready-made packs: packs.filesystem_safety, no_exfiltration, spend_cap (fails closed on missing or non-numeric amounts), rate_limit, approval_gated, combined with packs.combine.

How it works

flowchart LR
    T["Trace<br/>Anthropic / OpenAI / OTel / dicts"] -->|adapters| S["AgentSession<br/>ordered ToolCalls"]
    S --> C["ContractSet.check"]
    C --> R["ContractReport<br/>verdicts, offending call,<br/>decision"]
    R -->|assert_all_pass| P["pytest / CI"]
    A["Agent proposes a call"] --> G["FastGuard.check_call"]
    G -->|allow| X["run tool, commit"]
    G -->|block / require_approval| H["refuse or ask a human"]

Each contract is a pure function from a session to a result; there is no model in the loop. GuardedSession re-runs that function over the whole session for each new call (O(N) per call). FastGuard instead keeps a small state machine per contract (counts, first offending index, gate state, a window of timestamps) and answers from that state. A randomized differential test checks that the two agree, call by call, on verdict, severity, message and offending index across 400 random specs. More detail in docs/architecture.md.

Performance

Apple M5 Pro, Python 3.11. Commands are in the repo; numbers will differ on your machine.

Batch checking, 7 contracts (pytest benchmarks/test_benchmark.py --benchmark-only):

Live checks, 5 safety contracts (python benchmarks/bench_live_guard.py):

The full quickstart script above (interpreter start, import, check, print) takes about 35 ms wall-clock.

How it compares

DeepEval's ToolCorrectnessMetric and RAGAS's ToolCallAccuracy also score tool calls without an LLM: they compare a run against a reference list of expected calls (optionally ordered, optionally with arguments). That works well when there is one right trajectory.

AgentSpec checks rules that every acceptable run must satisfy, so it needs no golden trajectory and can say things a reference list can't: "never call X", "at most 20 searches a minute", "every deploy needs an approval for that environment", "no writes outside /workspace". It can also enforce those rules before a call executes. It does not judge whether the answer is good; pair it with an output evaluator for that.

Tests and CI

git clone https://github.com/rakshithmuda22/agentspec && cd agentspec
pip install -e ".[dev]"
pytest tests/ examples/ -q          # 173 passed
python examples/why_agentspec.py
python demo/demo.py

173 tests across tests/ and examples/, all offline. CI (.github/workflows/ci.yml) runs them on Python 3.9–3.13 with pyflakes, both demos and the stress tests, then builds the wheel, runs twine check and installs it into a clean venv.

Limitations

• Rules see tool names and arguments, not meaning. A generic bash tool needs argument rules (where=m.arg_matches("cmd", r"\brm\b")) to be useful.
• In a live guard, liveness rules (must_call, eventually, within_steps) fail until satisfied and so block early calls. Keep them in a separate end-of-run spec.
• FastGuard has incremental versions of must_not_call, must_call_at_most, must_precede / requires_approval, never_after / must_not_call_after and must_call. Other rules and custom add() checkers still run in O(N).
• path_under normalizes .. but does not resolve symlinks. URLs need a scheme for host rules to match (unparseable URLs are treated as not allowlisted).
• The case-study conversations are hand-written in Anthropic format, not recorded from a live model.

Roadmap

• agentspec check traces/*.json --spec spec.py CLI with JUnit XML output for CI.
• A LangGraph / LangChain callback adapter (today: from_dicts).
• Recorded traces from real model runs in the case study.

Project layout

agentspec/
  contracts.py     contract engine, reports, severities, decisions
  incremental.py   FastGuard (incremental live guard)
  runtime.py       GuardedSession, enforce(), Blocked / ApprovalRequired
  matchers.py      where= predicates (paths, hosts, args, combinators)
  packs.py         ready-made rule packs
  adapters/        anthropic.py, openai.py, generic.py (dicts + OTel spans)
  pytest_plugin.py fixtures + assert_satisfies / assert_violates
  recorder.py      mock sessions + an optional live Claude tool-use loop
tests/             unit, scenario, adapter and differential tests
examples/          why_agentspec.py, Anthropic-format case study
benchmarks/        pytest-benchmark, stress tests, live-guard benchmark

License

MIT. Built by Sai Rakshith Muda.