specscore

A CLI tool that scores OpenAPI specs for AI-readiness across 6 dimensions — giving APIs a letter grade and actionable recommendations so they work well with AI agents and LLM tooling.

Installation

# With uv (recommended)
uv pip install -e .

# Or with pip
pip install -e .

Usage

Scorecard

# Score an OpenAPI spec
specscore score path/to/openapi.yaml

# Output as JSON
specscore score path/to/openapi.yaml --json

# Run against the built-in sample spec to see how scoring works
specscore demo
specscore demo --json

Sandbox

The sandbox spins up a local mock server from your spec and auto-probes every endpoint, reporting a feasibility score — how well the spec can actually be exercised by an agent.

# Start a persistent mock server (press Ctrl+C to stop)
specscore sandbox start path/to/openapi.yaml
specscore sandbox start path/to/openapi.yaml --port 9000

# Probe all endpoints and get a report
specscore sandbox probe path/to/openapi.yaml
specscore sandbox probe path/to/openapi.yaml --json

# Run against the built-in sample spec
specscore sandbox demo
specscore sandbox demo --json

The probe command:

1. Starts an internal mock server
2. Generates synthetic request payloads and path parameters from the spec's schemas
3. Fires requests at every operation
4. Reports per-endpoint status codes, response times, and any issues found
5. Exits with code 2 if the feasibility score is below 50%

Exit codes

What gets scored

Each spec is evaluated across 6 weighted dimensions:

Grading

Example output

╭─ specscore · OpenAPI AI readiness ───────────────────────────╮
│  Task Manager API  v1.0.0                                      │
│  sample_spec.yaml                                              │
│                                                                │
│  Overall Score: 42.3/100  Grade: F                             │
╰────────────────────────────────────────────────────────────────╯

╭──────────────────────────────────┬───────────┬───────┬──────────────────────────┬────────╮
│ Dimension                        │     Score │ Grade │ Progress                 │ Issues │
├──────────────────────────────────┼───────────┼───────┼──────────────────────────┼────────┤
│ Foundational Compliance          │  75.0/100 │     C │ ███████████████░░░░░     │      1 │
│ Developer Experience             │  38.5/100 │     F │ ███████░░░░░░░░░░░░░░    │      3 │
│ AI-Readiness & Agent Experience  │  31.2/100 │     F │ ██████░░░░░░░░░░░░░░░    │      4 │
│ ...                              │       ... │   ... │ ...                      │    ... │
╰──────────────────────────────────┴───────────┴───────┴──────────────────────────┴────────╯

Issues & Recommendations

  AI-Readiness & Agent Experience
    x 4/6 operations lack descriptive intent (need >30 char description)
       paths.*.*.description
    ! 5/6 operations have no documented error responses (4xx/5xx) — agents cannot handle failure gracefully
       paths.*.*.responses

JSON output

Pass --json to get a machine-readable report:

specscore score openapi.yaml --json | jq '.overall_score'

{
  "api_name": "Task Manager API",
  "api_version": "1.0.0",
  "spec_path": "openapi.yaml",
  "overall_score": 42.3,
  "grade": "F",
  "dimensions": [
    {
      "name": "Foundational Compliance",
      "score": 75.0,
      "grade": "C",
      "issues": [...]
    }
  ]
}

Development

# Install with dev dependencies
uv pip install -e .

# Run directly without installing
python main.py score path/to/spec.yaml

License

The scorecard scoring model in this project (scorecard/ package) is derived from the Jentic API AI-Readiness Framework (JAIRF), which is licensed under the Apache License, Version 2.0. See the NOTICE file for the required upstream attribution.

Original contributions (sandbox, clitic, GitHub Pages site) are copyright (c) 2025-2026 SheepSeb. The project as a whole is distributed under the Apache License, Version 2.0 — see the LICENSE file for details.