agent-pr-evidence

Languages: English | 中文

Generate reviewable safety evidence for AI-agent-generated pull requests.

Status

P1 - v0.4.2 production hardening.

Purpose

Move Safe Agent Operations from config scanning into PR review and CI evidence.

First Production Surface

Local CLI and GitHub Action that produce a Markdown/JSON PR evidence packet from a git base/head diff. The GitHub App surface is deferred until the required permissions and real PR samples are available.

Install

From PyPI:

python3 -m pip install xone-agent-pr-evidence
agent-pr-evidence --version

From Homebrew:

brew install x-one-ai/tap/agent-pr-evidence
agent-pr-evidence --version

From this repository:

python3 -m pip install -e .
agent-pr-evidence --version

Usage

Collect local PR evidence from a git diff:

agent-pr-evidence collect --base origin/main --head HEAD --format markdown
agent-pr-evidence collect --base origin/main --head HEAD --format json --output pr-evidence.json
agent-pr-evidence collect --base origin/main --head HEAD --test-log pytest.log
agent-pr-evidence collect --base origin/main --head HEAD --config .agent-pr-evidence.yml --profile strict
agent-pr-evidence baseline --base origin/main --head HEAD --output agent-pr-evidence-baseline.json
agent-pr-evidence gate --base origin/main --head HEAD --baseline agent-pr-evidence-baseline.json --profile strict

The first production surface is local-first. It does not need GitHub App permissions and does not upload repository data.

Configuration

agent-pr-evidence automatically reads .agent-pr-evidence.yml from the repository root. Use --config to point at another file, and --profile to override the file for a single run.

schema_version: 1
profile: strict
disabled_risk_flags:
  - dependency-change

Profiles:

• default: lower-noise review evidence for teams adopting the tool.
• strict: adds missing-test-evidence when no test logs are provided.

Reports include schema_version: agent-pr-evidence.report.v1 so downstream workflow steps can check compatibility before consuming JSON.

Handoff Decisions

Markdown reports include a Suggested Next Step section after the handoff decision. It gives conservative reviewer actions, such as attaching passing test evidence, routing baseline review, blocking before merge, or creating a redacted failure packet with agent-failure-packet when test logs show a failed run.

Baseline Gate

Use a baseline when adopting the tool in an existing repository. The baseline records known risk flags, then gate fails only when a PR introduces new risk flags that are not already accepted.

agent-pr-evidence baseline --base origin/main --head HEAD --output agent-pr-evidence-baseline.json
agent-pr-evidence gate --base origin/main --head HEAD --baseline agent-pr-evidence-baseline.json --profile strict

Baseline files use schema_version: agent-pr-evidence.baseline.v1.

Rule Boundaries

Rules are backed by a versioned PR fixture corpus under tests/fixtures/pr-corpus. The corpus covers quiet changes, false-positive boundaries, and high-signal risks so rule changes can be reviewed before they affect real teams.

Current boundary examples:

• documentation-only changes stay quiet
• placeholder credentials such as <your-api-key> stay quiet
• real-looking token values still raise secret-like-content
• nested package manifests and lockfiles raise dependency-change
• agent instruction files such as AGENTS.md, .cursor/rules/*, .codex/*, .claude/*, and .cline/* raise agent-instruction-change

GitHub Action

Use the Action after actions/checkout with enough history for the base/head diff:

name: Agent PR Evidence

on:
  pull_request:

permissions:
  contents: read

jobs:
  evidence:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 0
      - uses: X-One-AI/agent-pr-evidence@v0.4.2
        with:
          base: ${{ github.event.pull_request.base.sha }}
          head: ${{ github.event.pull_request.head.sha }}
          output: agent-pr-evidence.md
          profile: strict
          baseline: agent-pr-evidence-baseline.json

The Action writes the report to GITHUB_STEP_SUMMARY and exposes report-path, summary-json, gate-failed, and new-risk-flags outputs. It does not request write permissions or post PR comments by default.

Required Evidence

• scope summary
• sensitive file changes
• test evidence
• dependency and CI/auth/infra changes
• reviewer checklist

Current Limits

• PR comments are intentionally not posted by default.
• GitHub App permissions are still skipped until real review workflows are available.
• Rule boundaries still need more real PR false-positive and false-negative tuning beyond the public fixture corpus.

Non-Goals

• not a generic code review bot
• not an autonomous merge gate
• not a hosted dashboard in the first version

OPT Operating Model

This project references the shared One Person Team workflow through ops/opt-overlay.md. Project-specific constraints live under ops/constraints, and evolvable local skills live under ops/skills.

Blocked Inputs

Inputs that require user or real-world data are recorded in ../x-one-skipped-inputs.md and should not block foundation work.

Real-user feedback should be classified as false-positive, false-negative, adapter-request, scenario-request, or catalog-update when it applies; portfolio-level handling is tracked in X-One portfolio health docs.

Docs

• Changelog
• Publishing
• Homebrew Packaging
• Example Config
• Example Baseline
• Reviewer Workflow
• Product Foundation
• OPT Overlay
• Production Constraints
• Main Entry Constraints
• Skill Evolution