DeployProof

Deterministic pre-push quality & security gate for modern Python codebases: In-memory AST mutation testing, credential scanning, and self-healing test synthesis. Built for human engineering teams and AI-assisted workflows alike.

📖 View the Complete Documentation & Interactive Product Portal

Why This Exists

Modern software development — whether crafted by human engineering teams or generated through AI coding assistants — introduces subtle failure modes that standard linters and line-coverage metrics completely miss:

• Deceptive Test Suites: Test suites boasting 90%+ line coverage that never assert true correctness, masking near-zero mutation scores.
• Accidental Credential Exposure: Hardcoded API keys, bearer tokens, or service credentials generated in passing or pasted into tests.
• Sandbox Escape Risks: Symlinks that deceive tool approval prompts into breaking outside the repository sandbox.
• Silently Swallowed Exceptions: Blanket except Exception: pass anti-patterns that hide critical runtime bugs.
• Dependency Hallucinations & Slopsquatting: Package names invented by LLMs or mistyped dependencies that don't exist on public PyPI.

DeployProof serves as an uncompromising, deterministic pre-push gate that validates code quality, test integrity, and security locally before any commit reaches CI or production.

Privacy & Security Guarantee: DeployProof runs 100% locally on your machine. It makes zero outbound network calls, except for querying the official PyPI registry (JSON API) to verify that newly introduced dependencies exist and are not hallucinated. DeployProof sends no source code, telemetry, test results, or secret findings to any external server.

Install

# Recommended — installs into an isolated environment, exposes the CLI globally
pipx install deployproof

# Or via pip into your current environment
pip install deployproof

Requires Python 3.10+. If you don't have pipx, install it with pip install pipx then run pipx ensurepath.

How to Use DeployProof

DeployProof provides two distinct modes of operation. We believe in total transparency about execution costs:

1. Diff-Scoped Pre-Push Gate (deployproof check) — Primary Fast Workflow

Evaluates only the files modified in your active session or git diff (1–3 files typically). Because only newly edited AST nodes are mutated, it executes in 2 to 5 seconds in local developer loops, pre-commit hooks, and pre-push gates.

# Fast daily check: verifies modified files in your current working tree / git diff (2-5s)
deployproof check

# Output structured JSON for automation or IDE tooling
deployproof check --json

# Enforce strict gates on newly introduced mocks or swallowed errors
deployproof check --strict-mocks --strict-error-handling

2. Full Repository Audit Mode (deployproof check --full-repo) — Thorough Codebase Audits

Evaluates every tracked Python file across the entire repository. Because mutation testing generates hundreds or thousands of mutants and runs the test suite against each one in isolated multi-worker sandboxes, full repository scans take real, honest compute time:

# Run a full-repository audit across all tracked files
deployproof check --full-repo

# Customize the parallel worker process count (defaults to auto-detected CPU count capped at 8)
deployproof check --full-repo --workers 8

Real-World Timing Expectations:

Parallel Multi-Worker Sandboxing (--workers N)

DeployProof includes a built-in multi-process execution engine (ProcessPoolExecutor) for parallel mutation testing. You can supply --workers <N> to both diff-scoped checks and full-repo audits:

# Parallelize a large uncommitted diff across 8 worker processes
deployproof check --workers 8

# Full repository audit distributed across 8 worker processes
deployproof check --full-repo --workers 8

How Parallel Sandboxing Works:

1. Snapshot Creation: DeployProof takes an initial atomic snapshot of your project into a clean temporary directory.
2. PID-Keyed Sandboxes: Each worker process receives its own dedicated filesystem sandbox (worker_<PID>), with an independent pytest cache (--override-ini=cache_dir=...) and separate temporary directory (--basetemp=...).
3. Zero Mutation Leaks: Mutants are generated and executed inside individual worker sandboxes in parallel. The working tree is untouched, and signal handlers (SIGINT, SIGTERM, SIGBREAK) ensure sandboxes are cleanly purged upon completion or interruption.

Advantages & When to Use:

• Large Diff / Full Repo Speedup: Near-linear execution scaling across CPU cores for batches with 50+ mutants, cutting 30-minute sweeps down to 5–8 minutes.
• Process & Cache Isolation: Eliminates cross-test state leakage, shared database locking, and .pytest_cache collisions between concurrent workers.

Trade-offs & When NOT to Use:

• Small Daily Diffs (1–3 files): Do NOT use --workers for small 2-line edits. Spawning isolated sandboxes and copying file trees incurs ~1–2 seconds of snapshot overhead. Sequential in-place mutation executes in 2–5 seconds with zero snapshot overhead.
• Disk Space & I/O Overhead: Running $N$ workers copies the repository snapshot $N$ times into temporary storage ($N \times \text{repo size}$ in tempfile.gettempdir()). On disk-constrained environments, use fewer workers (e.g. --workers 2 or --workers 4).
• Subprocess / Port Collisions: If your test suite binds to fixed network ports (e.g. localhost:8080) without dynamic port selection, parallel workers running tests concurrently may trigger port conflicts. Use isolated ports or run sequentially in such environments.

Hardware & Memory Sizing Architecture (RAM & CPU Optimization):

DeployProof's parallel sandboxing engine scales throughput directly with available physical RAM and logical CPU cores. Because test execution is CPU-bound and sandbox file I/O is memory-bound, hardware capacity dictates performance:

Total Memory Required ≈ Base OS Overhead (~2 GB) + [ N_workers × (Worker Process RSS + OS Page Cache Footprint) ]

1. Why More RAM Directly Maximizes Verification Speed:

• Zero-Latency In-Memory OS Page Cache: When physical RAM comfortably exceeds the aggregate working set of all $N$ workers, the operating system holds all sandbox file trees, compiled .pyc modules, and pytest test fixtures directly in the RAM page cache. File mutations and test imports achieve sub-millisecond execution with zero physical NVMe/SSD read/write contention.
• Elimination of Page-Fault Swapping: If total RAM is insufficient for the requested --workers N, the OS kernel is forced to page memory to disk (pagefile.sys on Windows or swap partitions on Linux). Page thrashing introduces severe disk queue latency that can degrade multi-process test throughput by 5× to 10×. Higher RAM guarantees that all workers remain 100% compute-active in physical memory.

2. Per-Worker Memory Consumption Profile:

• Python Runtime & AST Engine: ~35 MB RSS per worker.
• Pytest Test Suite & Dependencies: ~50 MB to 150 MB RSS per worker (depending on framework imports like FastAPI, Django, SQLAlchemy, or Requests).
• Sandbox Working Directory Snapshot: ~15 MB to 50 MB in OS file cache per worker.
• Total Allocation per Worker Process: ~100 MB to 250 MB RAM per worker.

3. Hardware Sizing & Safe Allocation Matrix:

4. Diff Scoped vs Full Repo Memory Comparison:

5. Minimum vs Recommended System Requirements:

• Absolute Minimum System RAM: 2 GB (for default sequential diff-scoped deployproof check).
• Minimum System RAM for Multi-Worker Mode (--workers 4): 4 GB.
• Recommended System RAM for Max-Throughput Parallel Mode (--workers 8 or 16): 16 GB (provides sufficient headroom to keep 8 to 16 Python subprocesses and their entire sandboxes resident in physical memory).

Output includes a section for each check — symlink scan, secrets scan, dependency scan, mock detection, control flow analysis, and mutation score — with a pass/fail line at the bottom.

Exit Codes:

• 0 — PASSED: All verification checks passed and mutation score meets threshold.
• 1 — FAILED: Code quality or security gate triggered (mutation score below threshold, untested files, hardcoded secrets, sandbox-escape symlinks, hallucinated packages, or strict flags).
• 2 — ERROR: Test environment failure (test suite failed to collect or execute before mutation testing began due to missing dependencies or broken imports).

Example Walkthrough

Given a newly written function calculator.py:

def calculate_discount(price: float, rate: float) -> float:
    if rate > 0.5:
        return price * 0.5
    return price * (1.0 - rate)

With an AI-generated test that achieves 100% line coverage by only asserting standard discounts (rate = 0.2):

def test_calculate_discount_basic():
    assert calculate_discount(100.0, 0.2) == 80.0

Running deployproof check mutates AST operators and detects that boundary conditions and threshold caps are untested:

$ deployproof check

DeployProof - LOCAL PRE-CHECK (approximate) - not the verified score
====================================================================

Target Scope (1 file evaluated):
  * calculator.py

Symlink & Sandbox Escape Scan (CWE-61/CWE-451):
  Clean: No symlinks or sandbox-escape traversal links detected across 1 session file.

Secrets & Credentials Pre-Push Scan:
  Clean: No hardcoded secrets or tracked .env files detected across 1 session file.

Dependency & Slopsquatting Scan (PyPI Registry & Age Analysis):
  Clean: No new external packages introduced across 1 session file.

Mock Usage Introduced (flagged for review):
  Clean: No modified test files in scope.

Control Flow & Error Handling (flagged for review):
  Clean: No bare excepts, swallowed exceptions, or unreachable code detected across 1 session file.

Local Pre-Check Mutation Verification:
  Score:  57.1% (4/7 mutants killed)
  Status: FAILED (score 57.1% below 80.0%) (threshold: 80.0%)
  Time:   2.27s

Skipped Constructs: None (No known unsupported constructs detected)

Surviving Mutants (3 unverified changes):

  [1] calculator.py:2
      Mutation: Replace numeric constant '0.5' with '1.5'
      Original: if rate > 0.5:
      Mutated:  if rate > 1.5:

  [2] calculator.py:3
      Mutation: Replace numeric constant '0.5' with '1.5'
      Original: return price * 0.5
      Mutated:  return price * 1.5

  [3] calculator.py:3
      Mutation: Replace binary operator '*' with '/'
      Original: return price * 0.5
      Mutated:  return price / 0.5

====================================================================
Notice: Local pre-check only. Full verified score runs in CI on push (via mutmut).
Pre-check FAILED: Score 57.1% is below threshold 80.0% (3 surviving mutants).

Adding tests for threshold cap (rate = 0.8) and exact boundary (rate = 0.5) kills all mutants:

Local Pre-Check Mutation Verification:
  Score:  100.0% (7/7 mutants killed)
  Status: PASSED (threshold: 80.0%)
  Time:   2.31s

Surviving Mutants: None (All generated mutants caught by test suite)

Pre-check clean: 100% of tested basic mutations caught.

Advanced Mutation Operators: DeployProof vs Mutmut vs Cosmic Ray

DeployProof features an enterprise-grade AST mutation engine specifically engineered for modern Python applications, AI-assisted development, and instantaneous pre-commit gates:

Enterprise AST SAST Scanner (OWASP Top 10 Coverage)

DeployProof includes a zero-dependency, high-precision AST SAST engine built natively into the pre-push gate, covering the most critical Python security risks:

Complete CLI Commands & Flags Reference

DeployProof provides a rich command-line interface with fine-grained control over every gate:

Core Commands

Mutation Testing & Self-Healing Options

Security & Quality Gates

Reporting, CI/CD & Configuration

Machine-Readable Output (--json)

DeployProof provides a stable structured JSON schema for CI/CD pipelines, IDEs, and automation:

deployproof check --json

JSON Output Schema

{
  "version": "1.1.11",
  "status": "passed",
  "summary": {
    "target_files_count": 1,
    "mutation_score": 100.0,
    "threshold": 80.0,
    "secrets_found": 0,
    "symlink_escapes_found": 0,
    "dependency_findings": {
      "high_risk": 0,
      "medium_risk": 0,
      "ok": 1,
      "unknown": 0,
      "unscanned": 0
    },
    "mock_usages_found": 0,
    "control_flow_findings": 0,
    "strict_mocks_active": false,
    "strict_mocks_triggered": false,
    "strict_error_handling_active": false,
    "strict_error_handling_triggered": false
  },
  "scope": {
    "target_files": [
      {
        "file": "app.py",
        "loc": 45,
        "is_large": false
      }
    ]
  },
  "mutation_testing": {
    "score": 100.0,
    "threshold": 80.0,
    "total_mutants": 6,
    "killed_mutants": 6,
    "survived_mutants_count": 0,
    "duration_seconds": 1.2,
    "surviving_mutants": [],
    "skipped_constructs": [],
    "untested_files": []
  },
  "secrets": {
    "clean": true,
    "files_scanned": 1,
    "findings": []
  },
  "symlinks": {
    "clean": true,
    "files_scanned": 1,
    "findings": []
  },
  "dependencies": {
    "clean": true,
    "total_scanned": 1,
    "findings": [],
    "unscanned_sources": []
  },
  "mocks": {
    "clean": true,
    "strict_gate_triggered": false,
    "findings": []
  },
  "control_flow": {
    "clean": true,
    "strict_gate_triggered": false,
    "findings": []
  }
}

What It Checks (The 7 Verification Gates)

• Gate 1: In-Memory Schemata Mutation Testing — Mutates AST operators (>=, ==, and, or, *, numeric constants, string boundaries) and switches mutants in warm RAM (__DEPLOYPROOF_MUTANT__), completely bypassing disk I/O.
• Gate 2: Actionable Self-Healing Test Synthesis — When mutants survive, --heal-tests and -i synthesize ready-to-run pytest unit tests with argument inference, class method instantiation, and boundary value checks to eliminate gaps.
• Gate 3: AST OWASP Top 10 SAST Scanner — Scans AST syntax trees for critical security flaws (SQL injection, command execution with shell=True, insecure deserialization, SSRF, hardcoded JWT keys).
• Gate 4: Secrets & 50-Commit Git History Scanner — Scans modified files and past 50 git commits for hardcoded API keys (OpenAI, Anthropic, AWS, GitHub, Stripe, private keys) and high-entropy secrets using Shannon entropy analysis.
• Gate 5: Dependency CVE & Slopsquatting Defense — Cross-references dependencies against the OSV vulnerability database and queries PyPI JSON API to detect hallucinated LLM packages.
• Gate 6: CWE-61 Symlink Sandbox Escape Gate — Resolves symbolic links and flags any whose target escapes the repository root (GhostApproval sandbox traversal defense).
• Gate 7: Control Flow & Swallowed Exceptions Gate — AST detector for bare except: without re-raise, silently swallowed exceptions (except Exception: pass), dead code, and mock leaks (--strict-mocks, --strict-error-handling).

Configuration (pyproject.toml)

DeployProof natively supports standard PEP 518 pyproject.toml configuration under [tool.deployproof]:

[tool.deployproof]
threshold = 85.0
workers = 8
timeout = 15.0
strict_mocks = true
strict_error_handling = true
sast_scanning = true
history_secrets_scanning = true
cve_scanning = true
generate_tests = "tests/test_deployproof_healed.py"

CI/CD & Pre-Commit Integration

GitHub Actions

Add .github/workflows/deployproof.yml to automatically emit PR inline annotations and visual Markdown step summaries:

name: DeployProof Gate
on: [push, pull_request]

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 50
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - run: pip install -e . && pip install deployproof pytest
      - run: deployproof check --github-actions --workers 4

Pre-Commit Framework (.pre-commit-config.yaml)

repos:
  - repo: https://github.com/SVSPraveen/DeployProof
    rev: v1.1.2
    hooks:
      - id: deployproof-check

Status & Roadmap

• Current (v1.1.2): In-Memory AST Schemata Mutation Testing, Actionable Self-Healing Test Synthesizer (--heal-tests), Interactive Quick-Fix Mode (-i), pyproject.toml [tool.deployproof] configuration engine, GitHub Actions native inline annotations and $GITHUB_STEP_SUMMARY dashboard, .pre-commit-hooks.yaml support, Full Repository Audit Mode (--full-repo) with isolated multi-worker sandboxes, AST OWASP Top 10 SAST scanner, 50-commit git history secrets scanner, OSV CVE database verification, GhostApproval symlink sandbox escape detector, 261 unit tests, and complete /docs product portal.
• Next: Reverse test-to-source dependency mapping (see FUTURE_SCOPE.md), SARIF 2.1.0 report exporter, and multi-language mutation rule packs.

Contributing

Issues and pull requests are welcome. Open an issue first for significant changes so the approach can be discussed before implementation.

License

MIT. See LICENSE.

Created & Architected by SVS Praveen · Portfolio · LinkedIn