mergeproof

Proof before merge. Pull requests earn their merge with evidence, not claims.

mergeproof is an evidence gate for pull requests. One YAML file says what a change must prove before it merges: tests for the modules it touched, a green integration job, before/after links from a live environment, a human who actually opened those links. CI enforces it. Coding agents read the same file, so the process stops living in review comments.

Why

Agents are good at claiming outcomes. "Added tests, verified on staging" costs nothing to write. Checklists don't help; a checkbox is self-attestation. What holds up is an artifact somebody else can open: a changed test file, a trace id that resolves, a screenshot pair, a sign-off tied to the exact commit. mergeproof checks those, verifies them against their source when it can, and leaves the last word to a person who is not the author.

Install

pip install mergeproof            # or: uv tool install mergeproof
pip install 'mergeproof[mcp]'     # adds the MCP server for coding agents

Or run it without installing anything. uvx fetches the package into a cache on first use; the container image on GHCR bundles Python, git and the mcp and otel extras:

uvx --from 'mergeproof[mcp]' mergeproof explain
docker run --rm -v "$PWD":/repo ghcr.io/aryamanz29/mergeproof check --local
docker run -i --rm -v "$PWD":/repo ghcr.io/aryamanz29/mergeproof mcp     # MCP over stdio, -i is required

Image tags follow the release tag: v1.2.3 publishes :1.2.3, :1.2, :1 and :latest; :edge tracks main. The image expects the repository mounted at /repo. For an MCP client, the same command goes in the settings file: "command": "docker", "args": ["run", "-i", "--rm", "-v", "/path/to/repo:/repo", "ghcr.io/aryamanz29/mergeproof", "mcp"].

Five-minute tour

mergeproof init          # writes a starter mergeproof.yaml
mergeproof checks        # what checks exist and what they take
mergeproof explain       # what must the current diff prove, what is missing right now
mergeproof template      # the evidence block still missing, ready to paste into the PR
mergeproof check         # the gate: exit 0 pass, 1 fail, 2 pending

explain on a branch that changed a tool module and nothing else:

# What this change must prove (fail)

## tool-change-needs-unit-tests [block]
Every changed tool module ships with changed unit tests for that module.
Triggered by `server/tools/search.py`.

- ❌ **unit tests for touched tools**: Each changed source file needs a changed test file:
  `server/tools/{name}.py` needs `tests/unit/**/test_{name}*.py`.
  - now: 1 changed source file(s) without test changes
  - server/tools/search.py: expected a changed test matching tests/unit/**/test_search*.py
  - fix: Add a regression test for each listed file: it should fail before the change and pass after.
- ⏳ **unit tests green**: CI check run matching `^unit` is green on the head commit.
  - now: CI status for `^unit` is only visible in GitHub mode

## fix-needs-live-evidence [block]
...
## Evidence block to add to the PR description

```evidence
environment: staging
image: <registry>/mcp-server:pr-<n>-<sha7>
traces:
- what: <what was exercised>
  before: https://langfuse.example.com/project/<project-id>/traces/<trace-id>
  after: https://langfuse.example.com/project/<project-id>/traces/<trace-id>
```

The policy file

version: 1
project: my-service

rules:
  - id: source-needs-tests
    description: Source changes ship with tests for the touched module.
    when:
      paths: ["src/**/*.py"]
      exclude_paths: ["src/**/__init__.py"]
    require:
      - check: tests.changed
        with:
          map: { "src/{pkg}/{name}.py": "tests/**/test_{name}*.py" }
      - check: ci.job_passed
        with: { name: "^unit", regex: true }

  - id: fix-needs-live-evidence
    description: Bug fixes show the behaviour before and after on staging.
    when:
      title: "^fix"
    instructions: Reproduce on staging, keep the link, deploy, repeat, keep that link too.
    require:
      - check: evidence.field
        with: { key: environment, equals: staging }
      - check: evidence.links
        with: { min_pairs: 1 }
      - check: review.human_verified
        with: { phrase: "/verified", bind_to_head: true }

A rule has a when (which changes it applies to) and a list of requirements, each a check with parameters. severity: warn on a rule or a requirement reports without blocking.

when matchers: paths, exclude_paths, labels, title (regex), base_branches, authors. That last one lets you hold copilot[bot] or claude[bot] to a stricter rule set than people.

Evidence lives in the PR description

A fenced YAML block. Agents can write it, machines can read it, humans can still skim it.

```evidence
environment: staging
image: registry.example.com/mcp-server:pr-77-4444444
traces:
  - what: search with empty query
    before: https://langfuse.example.com/project/p1/traces/trace-before-1
    after:  https://langfuse.example.com/project/p1/traces/trace-after-1
```

mergeproof template prints exactly the block a change still needs. Multiple blocks merge.

What the PR sees

One comment, updated in place on every push, label change, CI completion and review comment:

Wire it up with the action (copy examples/github-workflow.yml):

on:
  pull_request:
    types: [opened, synchronize, reopened, edited, labeled, unlabeled]
  issue_comment: { types: [created, edited] }   # /verified and agent verdicts land here
  check_suite:   { types: [completed] }         # re-check when CI finishes
jobs:
  gate:
    if: github.event_name != 'issue_comment' || github.event.issue.pull_request
    runs-on: ubuntu-latest
    permissions: { contents: read, pull-requests: write, statuses: write, checks: write }
    steps:
      - uses: actions/checkout@v4
        with: { ref: ${{ github.event.repository.default_branch }} }   # policy from the base branch
      - uses: Aryamanz29/mergeproof@v0.2.0
        env:
          MERGEPROOF_PR_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}

The action reports in three places, each switchable: the sticky comment above, a commit status named mergeproof in the merge box with the headline (3 of 5 requirements satisfied, 2 pending), and a Check Run in the Checks tab whose annotations land on the files concerned, for example "expected a changed test matching tests/**/test_search*.py" on server/tools/search.py. Require the mergeproof status in branch protection and the gate is enforced.

None of this blocks a merge until the repository says so. GitHub merges anything unless a ruleset requires specific checks (rulesets need a public repository or a paid plan), so create one that requires the mergeproof status next to the CI jobs you already trust:

gh api -X POST repos/OWNER/REPO/rulesets --input ruleset.json

{ "name": "main: pull requests with evidence", "target": "branch", "enforcement": "active",
  "conditions": { "ref_name": { "include": ["~DEFAULT_BRANCH"], "exclude": [] } },
  "rules": [
    { "type": "pull_request", "parameters": { "required_approving_review_count": 0,
        "dismiss_stale_reviews_on_push": true, "require_code_owner_review": false,
        "require_last_push_approval": false, "required_review_thread_resolution": false } },
    { "type": "required_status_checks", "parameters": { "strict_required_status_checks_policy": false,
        "required_status_checks": [ { "context": "mergeproof" }, { "context": "unit" } ] } } ] }

With pending-ok: "true" the workflow job stays green while evidence is outstanding, but the mergeproof status stays pending, and a pending required status blocks the merge button. That split is deliberate: the job says the tool ran, the status says whether the evidence is there.

Rendering with tools you already use

The action also writes the report as JUnit XML (mergeproof-junit.xml, one suite per rule, one case per requirement) and in reviewdog's format (mergeproof.rdjson, one diagnostic per file annotation). Any renderer for those formats then does the presentation:

      - uses: Aryamanz29/mergeproof@v0.2.0
        id: gate
      - uses: EnricoMi/publish-unit-test-result-action@v2     # rich check run: counts, per-requirement detail, trends
        if: always()
        with: { files: mergeproof-junit.xml, check_name: mergeproof requirements, comment_mode: off }
      - uses: reviewdog/action-setup@v1                       # inline review comments on the files concerned
      - run: reviewdog -f=rdjson -reporter=github-pr-review -level=error < mergeproof.rdjson
        env: { REVIEWDOG_GITHUB_API_TOKEN: ${{ github.token }} }

dorny/test-reporter reads the same JUnit file. Locally, mergeproof check -f junit and -f rdjson print the same documents.

By default all three appear under github-actions[bot] with GitHub's avatar. To have them show as mergeproof with the shield, create a GitHub App named mergeproof (avatar docs/logo.svg, permissions: pull requests write, checks write, commit statuses write), install it on the repo, and mint its token in the workflow:

      - uses: actions/create-github-app-token@v1
        id: app
        with:
          app-id: ${{ vars.MERGEPROOF_APP_ID }}
          private-key: ${{ secrets.MERGEPROOF_APP_KEY }}
      - uses: Aryamanz29/mergeproof@v0.2.0
        with:
          github-token: ${{ steps.app.outputs.token }}

The comment, the status and the Check Run are then attributed to the app, and a later push from the app's token does not trigger workflows recursively, which the default token would not either.

Built-in checks

mergeproof checks prints every parameter with its default.

Verifying links against their source

A pasted link should have to be real. evidence.links accepts a pattern with named groups and a verify name:

- check: evidence.links
  with:
    key: traces
    pattern: "^(?P<host>https?://[^/]+)/project/(?P<project>[^/]+)/traces/(?P<trace_id>[\\w-]+)"
    verify: langfuse

The core ships http (the URL answers 2xx; verify_options: {auth_header_env: DASH_TOKEN} for private dashboards). Anything vendor-specific is a plugin: a class with verify(url, match), registered under the mergeproof.verifiers entry-point group. examples/plugins/mergeproof-langfuse is a complete one in forty lines, for Langfuse, an MIT-licensed tracer you can self-host and feed from OpenTelemetry. Jaeger and Arize Phoenix work the same way.

Non-deterministic reviewers

LLM review agents are witnesses, not the gate. Have yours post a comment:

```verdict
check: trace-review
verdict: pass
head: 4444444
confidence: 0.85
summary: the after-trace returns the empty page; the before-trace raises.
```

and require it:

- check: agent.verdict
  with: { name: trace-review, authors: ["github-actions[bot]"], min_confidence: 0.8 }

authors pins the verdict to the bot that runs the agent, bind_to_head (default) discards it on the next push, and pairing it with review.human_verified keeps a person in the loop. If your agent already speaks in labels (reviewed / review-failed), pr.labels consumes those.

For coding agents

Agents get the rules two ways. Both are generated from the policy, so what an agent reads is what CI enforces.

A Markdown section for AGENTS.md or CLAUDE.md:

mergeproof agent-prompt >> AGENTS.md

It lists every rule, when it applies, what satisfies each requirement, and the evidence block to fill in, followed by three rules for agents: never fabricate evidence, never post the human verification phrase, keep the evidence block plain YAML.

An MCP server over stdio, for agents that can call tools:

pip install 'mergeproof[mcp]'
mergeproof mcp --policy mergeproof.yaml        # what the client launches

Register it once per repository. Claude Code reads .mcp.json at the repo root; Cursor and other clients take the same command and arguments in their own settings file:

{
  "mcpServers": {
    "mergeproof": { "command": "mergeproof", "args": ["mcp", "--policy", "mergeproof.yaml"] }
  }
}

The server exposes six tools and one resource:

Resource mergeproof://policy is the policy file itself.

Everything is read-only against the working tree. The server cannot post comments, add labels, or produce the human sign-off; an agent can find out what to prove and check its own work, and that is all. A typical session:

sequenceDiagram
    participant A as Coding agent
    participant M as mergeproof mcp
    participant G as GitHub
    A->>M: explain()
    M-->>A: fail: tests missing for tools/search.py, evidence block missing
    A->>A: write the regression test, reproduce on staging, keep both trace links
    A->>M: evidence_block({environment, image, traces})
    M-->>A: fenced evidence block
    A->>M: check(pr_body)
    M-->>A: pending: only CI status and reviewer sign-off remain
    A->>G: open the PR with that description
    G->>G: CI runs mergeproof check --github --comment
    Note over G: a human opens the traces and posts /verified sha7

The CLI and the server answer the same questions:

Tracing the server itself

The MCP SDK wraps every tool call in an OpenTelemetry span. Install the otel extra and point the standard variables at any OTLP/HTTP receiver and those spans are exported; nothing is sent otherwise.

pip install 'mergeproof[mcp,otel]'

# Langfuse (self-hosted or cloud): basic auth with the project keys
export OTEL_EXPORTER_OTLP_ENDPOINT=https://langfuse.example.com/api/public/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic $(printf '%s:%s' "$LANGFUSE_PUBLIC_KEY" "$LANGFUSE_SECRET_KEY" | base64)"

# Jaeger or an OpenTelemetry Collector on the default OTLP/HTTP port
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

Each span is named after the MCP method and tool (tools/call explain) and carries the gen_ai.tool.name attribute, so a dashboard can answer "which agent asked what, and how often did check come back failing" without any code in this project knowing which backend it talks to.

Tracing the server itself

The MCP SDK wraps every tool call in an OpenTelemetry span. Install the otel extra and point the standard variables at any OTLP/HTTP receiver and those spans are exported; nothing is sent otherwise.

pip install 'mergeproof[mcp,otel]'

# Langfuse (self-hosted or cloud): basic auth with the project keys
export OTEL_EXPORTER_OTLP_ENDPOINT=https://langfuse.example.com/api/public/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic $(printf '%s:%s' "$LANGFUSE_PUBLIC_KEY" "$LANGFUSE_SECRET_KEY" | base64)"

# Jaeger or an OpenTelemetry Collector on the default OTLP/HTTP port
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

Each span is named after the MCP method and tool (tools/call explain) and carries the gen_ai.tool.name attribute, so a dashboard can answer "which agent asked what, and how often did check come back failing" without any code in this project knowing which backend it talks to.

Plumbing

The porcelain commands are compositions of filters that read and write JSON:

mergeproof context --github > pr.json            # snapshot a PR (files, body, labels, comments, check runs)
mergeproof check --context pr.json -f json > report.json
mergeproof report report.json -f md              # render later, elsewhere
mergeproof comment report.json                   # post it

Snapshots make policies testable: the examples ship scenario files that the integration suite runs through the CLI, asserting the verdict each one claims.

Writing a check

from pydantic import BaseModel
from mergeproof import Check, Context, Outcome, Status


class ImageSmokeTested(Check):
    id = "image.smoke_tested"
    description = "The image named in the evidence block was exercised against the environment."

    class Params(BaseModel):
        registry: str

    def run(self, ctx: Context, params: Params, files: list[str]) -> Outcome:
        tag = ctx.evidence().get("image")
        if not tag or not tag.startswith(params.registry):
            return Outcome(
                status=Status.FAIL,
                summary="no image under the expected registry",
                fix="Build the branch image and record its tag under `image`.",
            )
        # query your deployment API here
        return Outcome(status=Status.PASS, summary=f"{tag} smoke-tested")

    def explain(self, params: Params) -> str:
        return f"An image under `{params.registry}` recorded as `image` and smoke-tested."

    def evidence_template(self, params: Params) -> dict:
        return {"image": f"{params.registry}/<name>:pr-<n>-<sha7>"}

[project.entry-points."mergeproof.checks"]
"image.smoke_tested" = "mypkg.checks:ImageSmokeTested"

Prior art

Danger runs rules written in JavaScript and comments on the PR; mergeproof is declarative, has a notion of evidence, and explains itself to agents. policy-bot enforces rich approval policies but only sees GitHub-native data. Checklist actions fail on unticked boxes, which is the thing agents will tick. Hosted "evidence gate" products evaluate your artifacts on their servers. mergeproof borrows policy-bot's matchers and Danger's sticky comment and runs entirely in your CI.

Exit codes

Development

make setup            # uv sync, example plugin, pre-commit hooks
make lint typecheck test integration
make check            # this repository's own gate against your working tree

MIT. See CONTRIBUTING.md.