Verdity — AI-Powered Pull Request Review System

Verdity — from verdict + fidelity/integrity. Every finding is a verdict (structured, evidence-backed, confidence-scored), delivered with integrity (calibrated, gated, auditable, never auto-posted without earning it).

⚠️ Security Notice

This system handles source code, secrets, and security findings. See SECURITY.md for the full threat model and hardening guide. Key controls:

• HMAC-SHA256 verified over raw body with constant-time comparison — no bypass, ever.
• Dual-secret rotation support with grace period.
• Delivery-ID replay cache with 24h TTL eviction.
• All secrets from environment / KMS — never committed.
• Append-only audit log with SHA-256 integrity checksums.
• Schema-validated output — agents cannot inject arbitrary commands; findings are structured data, not executable instructions.

Table of Contents

• Architecture
• Non-Negotiable Constraints
• Project Structure
• Quick Start
• Security
• Testing
• Configuration
• Build Phases

Architecture

[GitHub] ──(HTTPS)──▶ [Ingestion Gateway] ──▶ [Event Queue] ──▶ [Orchestrator]
                                              │                       │
                                              │               ┌───────┴───────┐
                                              │               ▼               ▼
                                              │         [Semantic Index]  [Specialists]
                                              │               │           (parallel)
                                              │               │              │
                                              │               ▼              ▼
                                              │         [Aggregator]   [Coding Agent]
                                              │               │              │
                                              │               ▼              ▼
                                              │         [Router]     [Verification Gate]
                                              │               │              │
                                              │               ▼              ▼
                                              │         [Approval Queue] [Regression Runner]
                                              │               │
                                              └───────────────┼──▶ [Audit Store]
                                                              ▼
                                                        [Token Economics]

Components

Non-Negotiable Constraints

Violating any of these is a build failure, not a style choice.

Project Structure

verdity/
├── src/verdity/
│   ├── __init__.py                 # Package root, version 0.3.0
│   ├── config.py                   # pydantic-settings; all secrets from env/KMS
│   ├── async_sqlite.py             # stdlib sqlite3 wrapped with asyncio
│   ├── schemas/
│   │   ├── __init__.py             # Re-exports (Severity, Finding, etc.)
│   │   └── _models.py              # All Pydantic data models
│   ├── hmac_verify.py              # HMAC-SHA256 + dual-secret rotation
│   ├── webhook_normalizer.py       # GitHub event → TriggerType
│   ├── event_queue.py              # SQLite-backed durable queue
│   ├── audit_store.py              # Append-only audit log with sha256 checksums
│   ├── token_economics.py          # Per-call metering + budget enforcement
│   ├── semantic_index.py           # Shared embeddings + symbol graph + incremental indexing
│   ├── orchestrator.py             # Durable workflow with scatter-gather fan-out
│   ├── aggregator.py               # Dedup, conflict resolution, ranking
│   ├── router.py                   # Confidence scoring + routing decisions
│   ├── approval_queue.py           # Persistent approval queue store
│   ├── coding_agent.py             # Deterministic fix generation + agentic fix mode
│   ├── review_rules.py             # Custom review rules (.verdity/rules.yml)
│   ├── mcp_server.py               # Model Context Protocol server (8 tools)
│   ├── verification_gate.py        # Gate checks + independent verifier + regression
│   ├── budget_enforcer.py          # Spend monitoring + degradation signals
│   ├── agents/
│   │   ├── __init__.py
│   │   ├── security.py             # Security specialist (3-pass scan)
│   │   ├── code_quality.py         # Code quality specialist
│   │   ├── testing.py              # Testing specialist
│   │   └── documentation.py        # Documentation specialist
│   └── gateway/
│       └── app.py                  # FastAPI ingestion gateway
├── tests/
│   ├── conftest.py                 # Shared fixtures (temp DB dirs)
│   ├── test_hmac_verify.py         # 11 tests
│   ├── test_webhook_normalizer.py  # 5 tests
│   ├── test_event_queue.py         # 7 tests
│   ├── test_audit_store.py         # 4 tests
│   ├── test_token_economics.py     # 7 tests
│   ├── test_semantic_index.py      # 7 tests
│   ├── test_gateway.py             # 5 tests
│   ├── test_orchestrator.py        # 16 tests
│   ├── test_phase4.py              # 7 tests
│   ├── test_phase5.py              # 11 tests
│   ├── test_phase6.py              # 14 tests
│   ├── test_phase7.py              # 5 tests
│   ├── test_phase8.py              # 13 tests (STRIDE checklist + e2e)
│   ├── test_mcp_server.py          # 11 tests
│   ├── test_review_rules.py        # 17 tests
│   └── test_semantic_index_full_context.py  # 11 tests
├── dev-notes/
│   ├── GOAL.md                     # Build charter (single source of truth)
│   ├── 01-Product-Requirements-Document.md
│   ├── 02-Technical-Architecture-Document.md
│   ├── 03-API-Webhook-Specification.md
│   ├── 04-Security-Threat-Model.md
│   ├── 05-Agent-Orchestration-Design.md
│   ├── PHASE-0{1..8}-GATE.md       # 8 gate validation documents
│   └── SECURITY-AUDIT.md           # This phase's security audit findings
├── .env.example                    # Template — NEVER commit real secrets
├── .gitignore                      # Excludes .env, *.db, __pycache__, .htmlcov
├── pyproject.toml                  # Project config, pytest settings
└── README.md                       # This file

Quick Start

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # or .venv\Scripts\activate on Windows

# Install dependencies
pip install -e ".[dev]"

# Set required environment variables
cp .env.example .env
# Edit .env — NEVER commit the real file

# Run the full test suite (requires 100% coverage)
pytest -v

# Start the gateway (development)
uvicorn verdity.gateway.app:app --reload --port 8000

Required Environment Variables

Security

See SECURITY.md for the full threat model, hardening guide, and STRIDE analysis.

Security Controls Implemented

Reporting Security Issues

Please report security vulnerabilities via GitHub Security Advisory, not via public issues.

Testing

# Run all tests with coverage (fails if <100%)
pytest -v

# Run a single test file
pytest tests/test_hmac_verify.py -v

# Run with coverage report
pytest --cov=src/verdity --cov-report=html
open htmlcov/index.html

Test Coverage

100% coverage enforced across all 342 tests in 15 test files.

Configuration

All configuration is via environment variables (see .env.example). In production, secrets should be injected from a managed secret store (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault) — never from .env files.

Database Paths

All SQLite databases default to :memory: for tests. For persistent storage:

QUEUE_SQLITE_PATH=/var/lib/verdity/queue.db
AUDIT_SQLITE_PATH=/var/lib/verdity/audit.db

Test databases are automatically created in a temp directory and cleaned up after each test run.

Build Phases

Total: 342 tests passing, 100% coverage enforced.

Distribution

Install from PyPI

pip install verdity

Build from source

git clone https://github.com/Dream-Pixels-Forge/verdity.git
cd verdity
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
pytest -v

License

MIT License — see LICENSE.