DeployProof

A pre-push verification tool for AI-assisted codebases: mutation testing, credential scanning, sandbox-escape detection, and dependency hallucination checks.

Why This Exists

AI-assisted development introduces failure modes that standard tools miss: test suites with high line coverage but near-zero mutation scores, hardcoded credentials generated in passing, symlinks that deceive approval prompts into escaping the repository sandbox, and package names hallucinated by LLMs that don't exist on PyPI. DeployProof catches these at the pre-push stage, before they reach CI or production.

Install

pip install deployproof

Requires Python 3.10+.

Quickstart

Initialize in your repository:

deployproof init

Run all verification checks against changes in the current session:

deployproof check

Output includes a section for each check — symlink scan, secrets scan, dependency scan, mock detection, control flow analysis, and mutation score — with a pass/fail line at the bottom. Exit code is non-zero on any finding that should block a push. Supports --json for machine-readable output and --strict-mocks / --strict-error-handling to turn informational checks into hard verification gates.

Machine-Readable Output (--json)

DeployProof provides a stable structured JSON schema for CI/CD pipelines, IDEs, and automation:

deployproof check --json

JSON Output Schema

{
  "version": "0.1.5",
  "status": "passed",
  "summary": {
    "target_files_count": 1,
    "mutation_score": 100.0,
    "threshold": 80.0,
    "secrets_found": 0,
    "symlink_escapes_found": 0,
    "dependency_findings": {
      "high_risk": 0,
      "medium_risk": 0,
      "ok": 1,
      "unknown": 0,
      "unscanned": 0
    },
    "mock_usages_found": 0,
    "control_flow_findings": 0,
    "strict_mocks_active": false,
    "strict_mocks_triggered": false,
    "strict_error_handling_active": false,
    "strict_error_handling_triggered": false
  },
  "scope": {
    "target_files": [
      {
        "file": "app.py",
        "loc": 45,
        "is_large": false
      }
    ]
  },
  "mutation_testing": {
    "score": 100.0,
    "threshold": 80.0,
    "total_mutants": 6,
    "killed_mutants": 6,
    "survived_mutants_count": 0,
    "duration_seconds": 1.2,
    "surviving_mutants": [],
    "skipped_constructs": [],
    "untested_files": []
  },
  "secrets": {
    "clean": true,
    "files_scanned": 1,
    "findings": []
  },
  "symlinks": {
    "clean": true,
    "files_scanned": 1,
    "findings": []
  },
  "dependencies": {
    "clean": true,
    "total_scanned": 1,
    "findings": [],
    "unscanned_sources": []
  },
  "mocks": {
    "clean": true,
    "strict_gate_triggered": false,
    "findings": []
  },
  "control_flow": {
    "clean": true,
    "strict_gate_triggered": false,
    "findings": []
  }
}

What It Checks

• Mutation Score — Mutates AST operators (>=, ==, and, or, *, numeric constants, comparisons) in modified files and runs your test suite against each mutant. Reports surviving mutants and a percentage score. Does not use line coverage.
• Secrets and Credentials — Scans modified files for hardcoded API keys (OpenAI, Anthropic, AWS, GitHub, Stripe, private keys) and tracked .env files using pattern matching and entropy analysis.
• Symlink and Sandbox Escape — Resolves symbolic links and flags any whose target escapes the repository root (CWE-61 / CWE-451). Catches the class of path-traversal trick used in the GhostApproval disclosure (Wiz Research, July 2026).
• Dependency and Slopsquatting — For each new import, dynamic import (importlib.import_module, __import__), or manifest entry (including recursive -r includes) introduced in the diff, queries the PyPI JSON API and checks registration age. Packages that don't exist (HTTP 404) are flagged HIGH RISK; packages registered within the last 30 days are flagged MEDIUM RISK.
• Mock Usage Detection — Scans test diffs for newly introduced imports or fixture uses of unittest.mock, mocker, and monkeypatch, flagging them for human review with an optional --strict-mocks hard gate.
• Control Flow and Error Handling — AST-based detector for bare except: without re-raise, silently swallowed broad exceptions (except Exception: that only pass or log/print without re-raising or returning error indicators), and dead/unreachable code following unconditional return, raise, break, or continue, with an optional --strict-error-handling hard gate.

What This Doesn't Do

• Not a full-repo audit. Checks are scoped to files changed in the current session (git diff). Files you haven't touched are not re-evaluated.
• Python only. Mutation testing and import extraction currently support Python files only. Other languages are not scanned.
• No auto-fix. DeployProof reports findings; it does not modify your code, rewrite imports, or suggest patches.
• No IDE plugin yet. There is no VS Code extension or JetBrains plugin. The CLI is the interface. IDE integration is on the roadmap.

See It Catch Real Bugs

Clone this repository and run the standalone stress-test suite to see DeployProof evaluate 7 planted edge cases:

python stress_fixtures/run_stress_tests.py

Fixtures cover weak test suites, zero-test orphan modules, planted OpenAI/AWS credentials, and GhostApproval sandbox-escape traps.

Status & Roadmap

• Current (v0.1.5): Diff-scoped AST mutation testing, secrets scanner (including unquoted .env values), GhostApproval symlink sandbox-escape detector, PyPI dependency hallucination / slopsquatting scanner (with recursive -r requirements scanning and dynamic import detection via importlib / __import__), mock-introduction detector (--strict-mocks), control-flow / swallowed-exception scanner (--strict-error-handling), machine-readable --json output, and Tier 2 CI verification via GitHub Actions (mutmut).
• Next: Multi-language mutation support and expanded ecosystem rule packs.

Contributing

Issues and pull requests are welcome. Open an issue first for significant changes so the approach can be discussed before implementation.

License

MIT. See LICENSE.

Created by SVS Praveen · Portfolio · LinkedIn