SpecFix — Speculative Sandbox Framework Using State-Graph Rollback Checkpoints for Automated Code Repair

Autonomous AI Bug Repair with a 0% Production Regression Guarantee. Every proposed patch is executed inside an isolated speculative sandbox and verified against test suites before committing. Failed patches trigger automatic Git rollbacks and feed detailed error diagnostics back into subsequent AI retry attempts.

---

📌 Executive Overview

Software engineering teams spend 30–50% of their time on manual bug triage and repair. While modern Large Language Models (LLMs) can generate code fixes, applying AI-generated patches directly to software repositories introduces significant risk: hallucinated code, broken syntax, or unexpected side-effects can corrupt production codebases.

SpecFix eliminates this risk by introducing a Speculative Sandbox Framework:

1. Speculative Execution: No AI patch is ever committed directly. Every fix is treated as an unverified speculation.
2. Git Rollback Checkpoints: Before applying a patch, SpecFix tags a rollback checkpoint (git tag). If verification fails, the repository is instantly reset (git reset --hard & git clean -fdx).
3. Automated Verification: Patches are tested in an isolated sandbox running the target project's automated test suite (pytest).
4. Self-Correction Retry Loop: Test failure outputs (stderr) are captured and injected into retry prompts, enabling AI models to self-correct patch alignment or syntax errors on subsequent attempts.
5. State-Graph Audit DAG: Every attempt—successful or failed—is stored as a node in a SQLite-backed Directed Acyclic Graph (DAG), providing full operational auditability.
6. Tiered Model Routing: Simple bugs are routed to lightweight Tier 1 SLMs (e.g., NVIDIA Qwen 2.5 Coder 32B), while complex multi-file bugs escalate to Tier 2 Cloud LLMs (e.g., Google Gemini 2.5 Flash), optimizing latency and API compute cost.

---

🏗️ System Architecture

                                SYSTEM PIPELINE ARCHITECTURE
 ┌────────────────┐     ┌────────────────┐     ┌─────────────────────┐     ┌──────────────────┐
 │ Bug Report &   │────▶│ Intake Gateway │────▶│ YAGNI Discipline    │────▶│ Complexity       │
 │ Target Repo    │     │ [intake.py]    │     │ Gate [yagni.py]     │     │ Analyzer         │
 └────────────────┘     └────────────────┘     └─────────────────────┘     │ [complexity.py]  │
                                                                           └────────┬─────────┘
                                                                                    │
 ┌────────────────┐     ┌────────────────┐     ┌─────────────────────┐              │
 │ State Graph    │◀────│ Verifier       │◀────│ Sandbox Execution   │◀─────────────┴──────────┐
 │ Audit DAG      │     │ (pytest)       │     │ Timeout Protection  │                         │
 │ [state_graph.py]     │ [verifier.py]  │     │ [sandbox.py]        │                         ▼
 └────────────────┘     └───────┬────────┘     └─────────────────────┘             ┌──────────────────┐
                                │                         ▲                        │ Tiered LLM Router│
                                │ (Verification Fail)     │ (Apply Patch)          │ [router.py]      │
                                ▼                         │                        └────────┬─────────┘
                        ┌─────────────────────────────────┴──┐                              │
                        │ Checkpoint Rollback & Retry Loop   │◄─────────────────────────────┘
                        │ [checkpoint.py]                    │ (Tier 1: NVIDIA Qwen /
                        └────────────────────────────────────┘  Tier 2: Google Gemini)

---

✨ Key Features

• 🛡️ Zero-Commit Guarantee: Zero broken or unverified code reaches production branches.
• 🔄 Automatic Rollback & Recovery: Clean Git reset mechanics restore pristine repository state on test failure.
• 🎯 YAGNI Prompt Discipline: Constrains LLMs to produce minimal, surgical unified diffs without unnecessary refactoring.
• 🩹 Robust Diff Normalization: Intelligently handles bare diff hunks, missing line counts, and hunk header formatting variations.
• 🧠 Deterministic Complexity Analyzer: Scores bugs based on stack depth, file counts, and concurrency keywords to determine routing tier.
• ⚡ Tiered LLM Routing:
  • Tier 1 (Local / Hosted SLM): NVIDIA Qwen 2.5 Coder 32B Instruct via NVIDIA API.
  • Tier 2 (Cloud LLM): Google Gemini 2.5 Flash API.
  • Offline Mock Fallback: Automatic fallback to deterministic MockModelProvider when API keys are not set.
• 📊 SQLite State-Graph Audit DAG: Comprehensive attempt tracing with parent-child retry relationships, exit codes, stderr feedback, and generated diffs.
• 💻 React + TypeScript Web Dashboard: Modern dark-mode web application featuring an Interactive Repair Workbench, Benchmark Evaluation Matrix, and State-Graph DAG Visualizer.

---

⚡ Quick Start

Prerequisites

• Python: 3.11 or higher
• Git: Installed and accessible in environment PATH
• Node.js: v18+ (Required only for running the Web Dashboard UI)

1. Installation

Install SpecFix from PyPI / local package wheel:

pip install specfix

For development or contributing:

# Clone repository
git clone https://github.com/your-username/SpecFix.git
cd SpecFix

# Create & activate virtual environment
python -m venv .venv
.\.venv\Scripts\activate   # Windows PowerShell
# source .venv/bin/activate  # Linux / macOS

# Install package in editable mode with dev dependencies
pip install -e ".[dev]"

2. Verify Installation

Run the automated unit test suite to confirm everything is set up correctly (no API keys required):

pytest

---

🗝️ Model Provider Configuration

SpecFix supports real AI model repair using Tier 1 (NVIDIA Qwen) and Tier 2 (Google Gemini) providers. Configure environment variables to enable real LLM execution:

Windows PowerShell

# Tier 1 Provider (NVIDIA Qwen API)
$env:NVIDIA_API_KEY="nvapi-your-key-here"
$env:NVIDIA_MODEL="nvidia/qwen-2.5-coder-32b-instruct"      # Optional default
$env:NVIDIA_BASE_URL="https://integrate.api.nvidia.com/v1"  # Optional default

# Tier 2 Provider (Google Gemini API)
$env:GEMINI_API_KEY="AIzaSy-your-key-here"
$env:GEMINI_MODEL="gemini-2.5-flash"                        # Optional default

Linux / macOS Bash

# Tier 1 Provider (NVIDIA Qwen API)
export NVIDIA_API_KEY="nvapi-your-key-here"
export NVIDIA_MODEL="nvidia/qwen-2.5-coder-32b-instruct"

# Tier 2 Provider (Google Gemini API)
export GEMINI_API_KEY="AIzaSy-your-key-here"
export GEMINI_MODEL="gemini-2.5-flash"

💡 Note: If API keys are omitted, SpecFix automatically falls back to MockModelProvider so tests and local trial runs execute seamlessly offline.

---

🚀 Running SpecFix

1. User Interactive Repair Workflow (specfix repair)

Navigate to any target Python project repository and launch the interactive repair wizard:

cd C:\Projects\MyProject

specfix repair

Or pass the problem description via CLI arguments non-interactively:

specfix repair --problem "divide() crashes on zero divisor. Fix divide() in calculator.py so zero divisor raises ValueError('Cannot divide by zero')"

Optional CLI flags:

• --repo PATH: Path to target repository directory (default: current working directory .).
• --problem TEXT: Non-interactive natural language problem description.
• --db PATH: Path to custom SQLite audit database (default: specfix.db).
• --retries N: Maximum repair retry attempts (default: 3).

2. Developer Benchmark Workflow (specfix run)

To evaluate automated repair against a JSON bug report:

specfix run --bug path/to/bug_report.json --repo path/to/clean_target_repo

2. Web Dashboard & API Server

SpecFix includes a full-featured React + TypeScript dark-mode Web Dashboard backed by a lightweight Python REST server.

Step 1: Launch Python REST API Backend

python server.py

(Runs REST API server on http://127.0.0.1:8000)

Step 2: Launch React Frontend (Development Mode)

In a separate terminal:

cd dashboard
npm install
npm run dev

(Launches interactive Web Dashboard UI at http://localhost:3000)

Web Dashboard Features:

• 🛠️ Automated Repair Workbench: Trigger repairs live on sample bug reports and view generated diffs & verification output.
• 📈 Benchmark Evaluation Matrix: View empirical test benchmark performance across bug categories.
• 🌳 State-Graph Audit DAG: Visualize attempt nodes, parent-child retry links, execution metrics, and error logs.

---

🧪 Empirical Evaluation Benchmark

SpecFix includes an empirical evaluation harness covering 4 representative bug categories:

Benchmark Bug Category	Class	Primary Provider	Success Rate	Attempt Recovery
Arithmetic / Logic	SIMPLE	Tier 1 (NVIDIA Qwen)	100% (PASS)	Attempt 1/2
Boundary / Off-by-One	SIMPLE	Tier 1 (NVIDIA Qwen)	100% (PASS)	Attempt 1/2
Data / Collection Logic	SIMPLE	Tier 1 (NVIDIA Qwen)	100% (PASS)	Attempt 1/2
Error-Handling Exception	SIMPLE	Tier 1 (NVIDIA Qwen)	100% (PASS)	Attempt 2/2 (Recovered via Error-Feedback Retry)

To run the evaluation suite locally:

python -m tests.evaluation.run_evaluation

---

📂 Project Structure

SpecFix/
├── pyproject.toml              # Package configuration, dependencies & CLI entrypoint
├── server.py                   # Python REST API server for Web Dashboard
├── README.md                   # Project documentation
├── PROJECT_PLAN.md             # Architectural specification & engineering plan
├── GUIDE_DEMO_GUIDE.md         # Guide presentation & live demonstration script
├── dashboard/                  # React + TypeScript + Vite + Tailwind CSS Web Dashboard
│   ├── src/                    # UI Components, Workbench, DAG visualizer, Benchmark Matrix
│   ├── package.json            # Node dependencies
│   └── vite.config.ts          # Vite build & dev server config
├── src/
│   └── specfix/
│       ├── __init__.py         # Package initialization
│       ├── __main__.py        # CLI entry point & main repair pipeline orchestrator
│       ├── intake.py          # Bug report parsing & Git repo state validation
│       ├── yagni.py           # YAGNI prompt discipline & unified diff validator/normalizer
│       ├── complexity.py      # Deterministic heuristic complexity analyzer
│       ├── router.py          # Two-tier model router (Tier 1 vs Tier 2)
│       ├── checkpoint.py      # Git rollback tag creation, reset & commit manager
│       ├── sandbox.py         # Isolated patch execution & timeout protection
│       ├── verifier.py        # VerificationResult dataclass & test runner output parser
│       ├── state_graph.py     # SQLite-backed attempt DAG audit database
│       ├── config.py          # Typed configuration thresholds & system defaults
│       └── llm/
│           ├── base.py        # Abstract ModelProvider interface, diff parser & repair helpers
│           ├── mock.py        # Deterministic mock provider for offline testing
│           ├── nvidia_qwen.py # NVIDIA Qwen API Provider (Tier 1 SLM)
│           └── gemini.py      # Google Gemini API Provider (Tier 2 LLM)
├── tests/                     # Comprehensive Pytest test suite (230+ unit & integration tests)
│   ├── test_checkpoint.py     # Git checkpoint creation & rollback tests
│   ├── test_complexity.py     # Complexity analyzer scoring tests
│   ├── test_intake.py         # Bug intake validation tests
│   ├── test_integration.py    # Full pipeline integration tests
│   ├── test_router.py         # Model router tier assignment tests
│   ├── test_sandbox.py        # Sandbox execution & timeout tests
│   ├── test_state_graph.py    # SQLite DAG state graph tests
│   ├── test_yagni.py          # YAGNI prompt enforcement tests
│   └── evaluation/            # Empirical benchmark suite runner & fixtures
└── scripts/
    └── setup_eval_targets.py  # Benchmark repository fixture setup utility