Parity

Parity analyzes behavior-defining AI changes in pull requests, discovers the most relevant existing eval target, validates the real coverage gaps, and proposes native eval additions that fit the target suite.

Parity is not an eval runner. It does not create or mutate hosted evaluator infrastructure. It reuses the eval system you already have.

What Parity Does

For each PR that changes prompts, instructions, guardrails, judges, validators, or similar behavior-defining assets, Parity:

1. Detects the behavioral change.
2. Resolves the best matching eval target and method.
3. Validates which gaps are actually uncovered.
4. Synthesizes native eval additions for that target.
5. Writes only native_ready evals after explicit approval.

Support

More detail: docs/platforms.md

Public Commands

These are the commands most users need:

• parity init — scaffold parity.yaml, the GitHub Actions workflow, and context/ stubs
• parity doctor — verify your setup and environment
• parity run-stage 1 — detect behavioral artifact changes in a PR
• parity run-stage 2 — analyze coverage gaps against existing evals
• parity run-stage 3 — synthesize native eval proposals
• parity write-evals — write approved evals to your platform after merge
• parity setup-mcp — generate an MCP server config from parity.yaml (for local agent tooling)

Internal CI commands (post-comment, resolve-run-id, etc.) are used by the generated workflow and are not intended to be called directly.

Quick Start

pip install parity-ai
parity init

Then:

1. Fill in the generated context/ files.
2. Add GitHub secrets: ANTHROPIC_API_KEY, OPENAI_API_KEY, and any platform keys you use.
3. Commit parity.yaml, .github/workflows/parity.yml, and context/.
4. Open a PR that changes AI behavior.
5. Add the fixed approval label parity:approve before merging if you want Parity to write approved evals back after merge.

Docs

• Configuration
• Architecture
• Platform support
• Example quickstart
• Maintainer guide

License

MIT