toolcall-linter

Local, offline, read-only linter that cross-checks agent transcript tool calls against declared schemas.

Install • Quick Start • Methodology • Formats • CI

---

What problem does this solve?

AI agents call tools hundreds of times per session. When an agent passes a wrong argument type, forgets a required parameter, or hallucinates a tool name, the failure often surfaces far downstream as a confusing runtime error. Debugging those failures means manually tracing through logs and comparing tool calls against schemas.

toolcall-linter makes those mistakes visible immediately, at lint time, with a single command.

toolcall-linter transcript.jsonl --tools tools.json

It reads the agent transcript, loads the declared tool schemas, validates every call against JSON Schema, and reports exactly where the contract is broken.

---

How it works

1. Parse the transcript — accepts Claude Code JSONL or OpenAI messages arrays.
2. Load the schema source — tools.json, an MCP server over stdio (mcp-stdio:<cmd>), or HTTP (mcp-http:<url>).
3. Match and validate — every tool call is looked up by name and validated with jsonschema.
4. Report violations — line-precise errors in text or JSON, with a nonzero exit code when anything fails.

---

What it catches

Failure class	Example	Why it matters
Undefined tool	{"name": "frobnicate"}	Agent invented a tool that does not exist in the schema.
Missing required argument	read_file without file_path	Runtime call will fail or read the wrong thing.
Wrong type	file_path: 123	String path expected, integer received.
Extra argument	read_file with encoding: "utf-8"	Schema forbids additional properties; agent is drifting.
Invalid enum value	output_mode: "full_text"	Agent picked a mode the tool does not support.

---

Install

# Recommended: install with pipx (isolated, global CLI)
pipx install .
pipx install git+https://github.com/Victorchatter/toolcall-linter

# Or in a local venv
python -m venv .venv
source .venv/bin/activate  # .venv\Scripts\activate on Windows
pip install -e .

No network calls, no telemetry, no credentials required.

---

Quick start

Create a tools.json with the schemas your agent is supposed to respect:

{
  "tools": [
    {
      "name": "read_file",
      "inputSchema": {
        "type": "object",
        "properties": {
          "file_path": { "type": "string" },
          "limit": { "type": "integer", "minimum": 1 }
        },
        "required": ["file_path"],
        "additionalProperties": false
      }
    },
    {
      "name": "Bash",
      "inputSchema": {
        "type": "object",
        "properties": {
          "command": { "type": "string", "minLength": 1 },
          "description": { "type": "string" }
        },
        "required": ["command", "description"],
        "additionalProperties": false
      }
    }
  ]
}

Create a transcript.jsonl that records what the agent actually did:

{"type": "tool_use", "name": "read_file", "input": {"file_path": "/etc/passwd"}}
{"type": "tool_use", "name": "read_file", "input": {"limit": 10}}
{"type": "tool_use", "name": "Bash", "input": {"command": "git status"}}
{"type": "tool_use", "name": "frobnicate", "input": {"x": 1}}

Run the linter:

toolcall-linter transcript.jsonl --tools tools.json

Output:

transcript.jsonl:2 read_file: 'file_path' is a required property
transcript.jsonl:3 Bash: 'description' is a required property
transcript.jsonl:4 frobnicate: tool 'frobnicate' is not declared in schema source

Exit code is 1 because violations were found.

Infer a schema from a tape

If you have an agent transcript but no declared tools.json, infer one from the tape:

toolcall-linter infer transcript.jsonl -o tools.json --pretty

The infer subcommand inspects every tool call, unions the argument keys seen for each tool, marks keys that appear in every call as required, infers JSON Schema types, and emits string enums when a property has ten or fewer distinct values. It writes an MCP-style tools.json and, by default, validates that schema against the source tape:

OK: inferred schema validates against 1 transcript(s)

Use --no-validate to skip the validation pass. Once you have tools.json, lint other transcripts with the usual command:

toolcall-linter another-transcript.jsonl --tools tools.json

You can pass multiple transcripts to infer to build a schema from a larger corpus:

toolcall-linter infer transcripts/*.jsonl -o tools.json --pretty

JSON output

toolcall-linter transcript.jsonl --tools tools.json --format json

{
  "ok": false,
  "violation_count": 1,
  "violations": [
    {
      "file": "transcript.jsonl",
      "line": 2,
      "tool": "read_file",
      "severity": "error",
      "message": "'file_path' is a required property",
      "schema_path": ""
    }
  ]
}

SARIF output

toolcall-linter transcript.jsonl --tools tools.json --format sarif > results.sarif

SARIF output includes runs[0].results with ruleId, message.text, and locations[0].physicalLocation mapped to the transcript file and line number. Use it with any SARIF-compatible CI viewer or the GitHub Actions workflow below.

---

Methodology

Fail fast at lint time

The guiding principle is to move the detection of contract violations as early as possible — from runtime debugging to a fast, deterministic lint step. Every agent transcript is a record of tool calls that should already satisfy the tool schemas. Treating it as a lintable artifact lets you:

• catch schema drift after prompt changes
• compare an agent's actual calls against its intended capability set
• gate CI on transcript correctness
• reproduce and bisect failures with exact line numbers

Schema-first validation

toolcall-linter does not guess what a tool should accept. It uses the actual declared JSON Schema (MCP inputSchema, OpenAI function parameters, or a tools.json file). This keeps the linter honest: if the schema is wrong, the linter is wrong in the same way, which forces the schema itself to become the source of truth.

Read-only and local

The tool never modifies the transcript, never sends data anywhere, and never calls the tools it inspects. This makes it safe to run on production transcripts containing file paths, queries, or other sensitive arguments.

---

Supported schema sources

Static tools.json

toolcall-linter transcript.jsonl --tools tools.json

Accepts either a JSON array of tools or an object with a "tools" key. Each tool must provide name and inputSchema (MCP style) or parameters (OpenAI style).

MCP server over stdio

toolcall-linter transcript.jsonl --tools "mcp-stdio:python -m my_mcp_server"

The linter spawns the command, sends a JSON-RPC tools/list request, and uses the returned schemas.

MCP server over HTTP

toolcall-linter transcript.jsonl --tools mcp-http:http://localhost:3000

Queries http://localhost:3000/tools/list and extracts the tool list from the response.

---

Supported transcript formats

Claude Code JSONL

Each line is a JSON object. The linter extracts tool calls from:

• { "name": "...", "input": {...} }
• { "tool_calls": [...] } with function.name and function.arguments
• legacy function_call objects

OpenAI messages array

A JSON file starting with [ is treated as an OpenAI messages array. The linter reads assistant messages containing tool_calls:

{
  "role": "assistant",
  "tool_calls": [
    {
      "id": "call_123",
      "type": "function",
      "function": {
        "name": "read_file",
        "arguments": "{\"file_path\":\"/etc/passwd\"}"
      }
    }
  ]
}

The arguments string is parsed as JSON before validation.

---

CLI reference

toolcall-linter <transcript>... --tools <source> [--format text|json]
toolcall-linter infer <transcript>... -o <tools.json> [--pretty] [--no-validate]

Lint

Argument	Description
<transcript>...	One or more paths to transcript files (JSONL or JSON array). Supports globs.
--tools <source>	Schema source: tools.json, mcp-stdio:<cmd>, or mcp-http:<url>. Required.
`--format text	json

Infer

Argument	Description
<transcript>...	One or more paths to transcript files to inspect. Supports globs.
-o, --output <tools.json>	Output file for the inferred MCP-style tool schema. Required.
--pretty	Write formatted JSON with indentation.
--no-validate	Skip validating the inferred schema against the source transcript(s).

Exit codes

Code	Meaning
0	No violations found.
1	One or more violations found.
2	CLI or configuration error (bad file path, malformed schema, etc.).

---

Terminal demo

---

CI integration

Use the nonzero exit code to block bad transcripts in CI:

# .github/workflows/lint-transcripts.yml
name: lint-transcripts
on: [push, pull_request]
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv tool install .
      - run: toolcall-linter transcripts/*.jsonl --tools tools.json

GitHub Actions with SARIF

Generate a SARIF file and upload it to GitHub Advanced Security so violations appear inline on the PR diff:

# .github/workflows/lint-transcripts-sarif.yml
name: lint-transcripts-sarif
on: [pull_request]
jobs:
  lint:
    runs-on: ubuntu-latest
    permissions:
      security-events: write
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv tool install .
      - run: toolcall-linter transcripts/*.jsonl --tools tools.json --format sarif > toolcall-linter.sarif
      - uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: toolcall-linter.sarif

Because the linter is read-only and offline, it is safe to run on every commit.

GitHub Action

The repo provides a reusable composite action at action.yml. It installs the linter from the checked-out repo, runs it on the supplied transcript(s), writes a report, and exposes the finding count as an output.

# .github/workflows/lint-toolcalls.yml
name: lint-toolcalls
on: [push, pull_request]
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Victorchatter/toolcall-linter@v0.4.0
        with:
          transcript: transcripts/*.jsonl
          tools: tools.json
          format: sarif
          fail-on-blockers: true

Inputs:

Input	Required	Default	Description
transcript	yes	—	Path to transcript(s). Glob patterns are supported.
tools	yes	—	Path to tools.json or MCP source.
format	no	sarif	text, json, or sarif.
fail-on-blockers	no	true	Fail the step when findings are reported.

Outputs:

Output	Description
findings-count	Number of findings reported.
report-path	Path to the generated report file.

pre-commit hook

Add the hook to .pre-commit-config.yaml:

repos:
  - repo: https://github.com/Victorchatter/toolcall-linter
    rev: v0.4.0
    hooks:
      - id: toolcall-linter
        args: [--tools, tools.json, --format, sarif]

The default hook runs on JSON files. Override files or types to lint committed transcripts. Pass --warn-only to report issues without blocking the commit.

---

Development

# Install in editable mode
python -m pip install -e .

# Run the synthetic end-to-end test
python selfcheck.py

Expected output:

Found 5 violations
 - 'file_path' is a required property
 - 123 is not of type 'string'
 - Additional properties are not allowed ('encoding' was unexpected)
 - tool 'frobnicate' is not declared in schema source
 - 'maybe' is not one of ['yes', 'no']
PASS

---

Design

The v1 design spec is available at:

• docs/superpowers/specs/2026-07-21-toolcall-linter-design.md

---

License

MIT. See LICENSE.