Covenant

A contract linter and drift firewall for MCP servers.

When an MCP server changes a tool — renames an output field, tightens an input schema — nothing throws. The LLM agents depending on that tool keep calling it, read a field that no longer exists, and confidently report a wrong answer. Existing MCP scanners hash tool definitions and tell you that something changed; Covenant classifies every change by whether it breaks the agent — and enforces the verdict, from CI to runtime.

Covenant makes the tool contract explicit, versioned, and enforced:

Quickstart

git clone https://github.com/Mhemd139/Covenant && cd Covenant
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

The repo ships a real example server with a committed baseline. Check it, then break it for real — COVENANT_DRIFT=1 renames a live tool's output field:

covenant check                    # OK no drift - exit 0
COVENANT_DRIFT=1 covenant check   # catches the lie - exit 1

The lie is caught twice: in the declared schema (output rows) and in the actual response body (behavior rows), because the committed config probes the tool.

Point it at your own server via covenant.toml, or inline:

pip install covenant-mcp
covenant snapshot --server http://localhost:8000/mcp   # or a stdio launch command
covenant check    --server http://localhost:8000/mcp --json

The severity model

The consumer of an MCP tool is an LLM agent that re-reads tool definitions on every run — which changes what "breaking" means. Covenant classifies by one direction principle:

Input-side changes fail loud — the server rejects the call, or the agent adapts → DEGRADED (warn; fail CI only under --strict). Output-side changes fail silent — the agent reads a value that is gone, retyped, or now null, and proceeds confidently → BREAKING (fail CI; quarantine at the proxy).

Nested schemas are walked recursively (balance.currency, items[].sku). Composed schemas ($ref/allOf/anyOf/oneOf) are never guessed at — a change there flags DEGRADED for manual review. Full rationale: Layer 0 design spec.

Use it in CI

Commit covenant.toml + covenant.lock.json, then:

- name: Contract check
  run: |
    pip install covenant-mcp
    covenant check --json   # exit 1 on breaking drift, 2 on config/connection error

This repo runs exactly that against its own example server on every push — including a job that injects the breaking change and asserts Covenant catches it (ci.yml).

Behavioral drift: probes + judge

A schema check can't see a server that lies — schema unchanged, response different. And most real MCP tools declare no outputSchema at all. Probes cover both: commit safe, read-only example calls in covenant.toml:

[[probes]]
tool = "get_transactions"
args = { account_id = "acct-001" }

snapshot stores each response's fingerprint (the type shape of what actually came back); check re-runs the probes and classifies shape drift with the same severity model.

A fingerprint remembers that a number lives there — not which number. When the exact value is part of the contract — a reference balance, a currency code, a unit — pin it:

[[probes]]
tool = "get_account"
args = { account_id = "acct-001" }
expect = { balance_usd = 4210.0, currency = "USD" }

check compares every pinned field against the live response with exact equality — no tolerance, no patterns. A mismatch is BREAKING: schema and shape still match while the value lies (a balance rescaled to cents, dollars quietly converted to another currency) — exactly the silent failure the direction principle exists to catch. Pins are opt-in and deterministic, like pip --require-hashes: nothing is pinned unless you type it. Try it on the example server — COVENANT_SEMANTIC_DRIFT=1 covenant check rescales the live balance ×100 and exits 1.

For drift you didn't pin — fields too volatile to pin, meaning shifts across the whole response — add the LLM judge:

pip install -e ".[judge]"
covenant check --judge    # [judge] model in covenant.toml: claude-* / gemini-*

Judge verdicts are advisory by design — DEGRADED, never BREAKING: a probabilistic detector must not trigger quarantine. Details: Layer 3 design spec.

Runtime guard: the proxy

The linter catches drift at ship time; the proxy contains it at runtime. It forwards every JSON-RPC exchange byte-for-byte (SSE passthrough included) — but a tools/call to a quarantined tool is short-circuited with a clean MCP isError result.

pip install -e ".[proxy]"
covenant proxy --upstream http://localhost:8000/mcp --port 9000
# point your MCP client at http://127.0.0.1:9000/mcp

Detection is proxy-owned: refresh re-lists the upstream itself, so enforcement never depends on the client's tools/list timing. Optional Postgres persistence keeps quarantine across restarts (--database-url, [store] extra); store writes are best-effort and never fail the request path.

docker compose up -d also brings up Prometheus + a provisioned Grafana dashboard at http://localhost:3000 — the quarantine stat flips green→red within one scrape of a drift:

Kubernetes: the MCPContract operator

Declare contract conformance instead of scripting it. The Helm chart ships the proxy, a kopf operator, and an MCPContract CRD — the operator re-runs the check on each contract's own schedule, writes the verdict into status, and nudges the proxy to quarantine on drift:

docker build -t covenant-mcp:0.1.0 .
helm install covenant deploy/helm/covenant --set proxy.upstream=http://my-server:8000/mcp
kubectl create configmap covenant-baseline --from-file=covenant.lock.json
kubectl apply -f examples/mcpcontract.yaml
kubectl get mcpcontracts -w        # RESULT flips clean -> breaking when the server drifts

A failed check is status.result: error, never a crash-loop. Details: Layer 5 design spec.

Architecture

Dependency-ordered layers; each ships alone, and every enforcement surface (CI, proxy, operator) reuses the same classifier and baseline format:

The full codebase tour lives in docs/ARCHITECTURE.md; per-layer design specs (rationale, rule tables, named decisions) in docs/specs.

Development

pip install -e ".[dev]"
pytest                        # Postgres-backed tests skip without COVENANT_TEST_DB
ruff check . && mypy covenant # mypy is strict

Layer boundaries are enforced by imports: the core depends only on mcp, typer, rich; everything else is an optional extra imported on use.

License

MIT