lap — how many tokens does your API cost an LLM agent?

Point lap at any OpenAPI spec, live MCP server, or your own agent config and get the token cost decomposed (definitions / call / result), a 0–100 grade, the concrete rule violations driving the cost — and an applicable patch for the fixable ones. Neutral, offline-capable, MIT.

pip install "lap-score[mcp]"

The five commands

lap stack                                  # YOUR installed MCP servers: "N tokens before you type a word"
lap score  openapi.json                    # A/B/C token decomposition + the LAP grade
lap lint   openapi.json                    # which rule violations drive the cost (CI-gateable; --discovery probes /llms.txt)
lap lint   --mcp "python -m mcp_server_git"   # ...same for a live MCP server (stdio or --mcp-url)
lap fix    openapi.json --apply patched.json  # the fixable findings as an OpenAPI Overlay patch

What that looks like:

$ lap stack
  server        kind   tools  menu tokens  compact
  time          stdio      2          283       31
  git           stdio     12         1418      153
  TOTAL                    14         1701      184
Your agent pays ~1,701 tokens of tool menus at session start - before you type a word.
Compact signatures of the same tools would cost 184 (+89% saved).

$ lap score api/openapi.json
LAP grade: B (72/100)   [menu 100  result 90  hygiene 0]
  variant       A tokens  saved vs full
  openapi_full  418       +0%
  compact_sig   205       +51%
  tool_search   163       +61%
Estimated call size (bucket B): mean ~18 tokens/call ...
Estimated result size (bucket C): GET /books ~465 tokens (list) -> ~305 if projection were added (R1)

$ lap fix api/openapi.json --apply patched.json
[written] lap-overlay.yaml  (6 action(s))
[written] patched.json  (lint findings: 15 -> 3)      # grade: B (72) -> A (91)

lap fix emits a standard OpenAPI Overlay 1.0.0 (R3 → limit param, R1 → fields, R2 → filter, E1 → declared 4XX) — it declares the contract; your server still implements it. lap badge <spec> writes a shields.io endpoint JSON so your README can carry the grade.

CI gates

Every command is --json-able and can fail a build:

lap score openapi.json --gate-form compact_sig --max-menu-tokens 800   # menu too heavy
lap score --diff old.json new.json --max-growth 500                    # this PR bloated the menu
lap score --diff --git HEAD~1 openapi.json --max-growth 500            # same, straight against a git ref
lap lint  openapi.json --fail-on warn                                  # rule violations (--ignore R2,A1 / .lapignore)

As a pre-commit hook (gates every commit that touches the spec):

# .pre-commit-config.yaml
repos:
  - repo: local
    hooks:
      - id: lap-menu-gate
        name: lap - agent-menu size gate
        entry: lap score --diff --git HEAD api/openapi.json --max-growth 500
        language: system
        files: ^api/openapi\.json$
        pass_filenames: false

…or the bundled composite Action:

- uses: lCrazyblindl/lap@v0.8.0
  with:
    spec: api/openapi.json
    max-menu-tokens: "800"
    fail-on: warn
    badge-path: docs/lap-badge.json   # optional: grade badge JSON

(Prefer Spectral? The same rules ship as a Spectral ruleset.)

Python API (stable from 0.7)

The CLI's data is available as plain Python — four functions, loose-semver-stable (signatures only break at a major version). Each accepts a file path, an http(s) URL, or an already-parsed spec dict:

import lap

report   = lap.score_spec("openapi.json")          # dict  = `lap score --json`
findings = lap.lint_spec("openapi.json")           # list[lap.Finding(rule, severity, where, message)]
grade    = lap.grade_spec("openapi.json")          # {"score": 72, "letter": "B", ...}
delta    = lap.diff_specs("old.json", "new.json")  # dict  = `lap score --diff --json`

MCP-side helpers (need pip install "lap-score[mcp]"): lap.mcp_client.fetch_tools(), lap.mcp_client.score_tools(), lap.lint.lint_tools(). Everything else under lap.* is internal and may change between minor versions.

Reading the numbers

• Bucket A (measured) — the tool-definition menu the model carries every session, under four renderings: naive OpenAPI→tools, compact signatures, numbered, lazy tool_search. With fastmcp installed, a real-MCP baseline row too.
• Buckets B / C (estimated from the schemas) — the call the model emits (required args in a tool-use envelope) and the result that comes back (per response schema, page-size aware, envelope-aware, real example values honored). Structural lower bounds. List responses also get a projected figure — what field projection would save, per endpoint.
• The grade — menu tokens/operation (weight 0.45) + heaviest result (0.30) + lint findings/operation (0.25), log-scaled; formula. Calibration: LaunchDarkly B, Spotify C, GitHub D, Google Drive F.
• Tokenizer — offline = tiktoken approximation (absolutes ≈, ordering robust: checked under 4 BPE vocabularies, Kendall τ ≥ 0.992). Set ANTHROPIC_API_KEY for faithful count_tokens figures.
• Parses OpenAPI 3.x and Swagger 2.0, YAML, allOf/oneOf/anyOf, $refs, non-JSON media types; crash-free across 175+ real APIs.guru specs.

The receipts

Every number and rule has a reproducible measurement behind it: the live leaderboard of 50 real APIs (naive menus total ~11.2M tokens; ~82% recoverable), the LAP profile (every rule cites its experiment), and the state of the field — which vendor claims we verified live, and which our measurements dispute. Issues/PRs: CONTRIBUTING (there's a "Score my API" issue template — disputes welcome).

Module map