plan-auditor

An independent, multi-layered verification supervisor for AI coding agents.

AI coding agents routinely leave work half-done or claim "done" when it isn't. Root cause: no explicit, machine-checkable plan, and nobody verifying the claims. plan-auditor fixes both — and goes further: it runs independently of the main AI, seals plan criteria against weakening, watches execution in real time, and refuses to count unverified work as done. It supports parallel / concurrent agents.

Two modes

Architecture

User Task
   │
   ▼
Main AI Agent ──────────────► proposes plan
   │                              │
   │                              ▼
   │                 ┌──────────────────────────────┐
   └───────────────► │  Independent Supervisor       │
                     │  (separate process / daemon)  │
                     └──────────────┬───────────────┘
                                    │
              ┌─────────────────────┼──────────────────────────┐
              ▼                     ▼                          ▼
     L0 Event Layer        L1 Requirements           L5 STRIPS-like
     (pattern detect)      (structured reqs)         Plan Verifier
              │                     │                          │
              ▼                     ▼                          ▼
     L2 World Model        L3 Policy Engine          L9 Watchdog
     (workspace state)     (deterministic rules)     (runtime observe)
              │                     │                          │
              └─────────────────────┼──────────────────────────┘
                                    ▼
                     L10 Deterministic Core (audit_check.py)
                                    │
                                    ▼
                     L8 Sealing + Monotonic Verification
                                    │
                                    ▼
                     L13 Completion Gate ──► PASS / FAIL / UNKNOWN
                                    │
                                    ▼
                     L12 Adversarial AI (optional semantic review)

See docs/architecture.md for the full data flow and layer responsibilities.

Layers

Subsumption priority (authoritative)

LEVEL 0  process / system safety        (supervisor alive, no deadlocks)
LEVEL 1  plan integrity                 (seal unchanged, monotonic)
LEVEL 2  deterministic verification     (L10 core, real checks)
LEVEL 3  security policy               (L3 rules: secrets, injection)
LEVEL 4  requirement coverage           (all reqs checked)
LEVEL 5  AI semantic / adversarial judgment (L12)

A failure at level N cannot be overridden by a PASS from any level > N.

Why independent verification?

An LLM auditing its own work with prose can hallucinate a pass. The evidence engine here is deterministic code: subprocess exit codes, filesystem facts, regex matches, pytest results. The main AI's judgment can only tighten the gate, never loosen it.

Parallel / multi-agent support

The supervisor is not single-agent. Multiple agents can work on the same workspace concurrently:

• Each plan step declares file ownership (owns, inferred from verify paths if absent).
• Overlap -> conflict warning (or block in STRICT profile).
• Heartbeats track liveness; stale ownership auto-releases.
• Lock files + append-only hashed registry coordinate agents.

Modes: serial | parallel-warn | parallel-strict.

Local-first / hardware-adaptive

Core verification never requires an LLM. The supervisor is fully useful in TIER 1.

Profiles

Install

One package, every agent. Copy the plan-auditor/ folder into the skills directory of whichever tool you use. No config, no build — the folder is the install.

How it auto-activates (no extra setup)

This works like any other skill the agent already has. Two activation paths:

1. Automatic via description matching (all tools)

Every tool reads SKILL.md's description field. When the user's request matches it, the skill loads automatically — the agent never needs to be told to use it. The current description triggers on phrases like "build X", "implement Y", "add a feature", "plan it", "don't leave it half-done", "did you actually do it".

Example flow:

You: "Build the login form with tests"
   │
   ▼ (Claude Code / Codex / Cursor / Grok / OpenCode sees the request
     matches the skill's description → loads ~/.../skills/plan-auditor/SKILL.md)
   │
   ▼ (Agent follows SKILL.md)
   │  1. Writes .plan-auditor/plan.json with concrete verify checks
   │  2. Implements each step
   │  3. After each step runs: python .../audit_check.py run <dir>
   │  4. Only finishes when: python .../audit_check.py audit <dir> exits 0
   │
   ▼
Agent: "audit PASSED — all steps verified"   ← not just "done"

No /plan-auditor invocation needed — it self-loads. You can also call it explicitly (/plan-auditor "build the login") for a one-shot task.

2. Platform hook (Command Code — unskippable)

Command Code additionally wires scripts/stop_gate.py as a Stop hook. At every turn-end, if any active plan has unverified steps, the turn is blocked and the model is told to run the auditor. The agent cannot finish, skip the audit, or relax a failed check. See docs/integrations.md for the one-block settings.json.

CLI (Supervisor Mode)

plan-auditor supervisor start  [--profile LIGHT|STANDARD|STRICT]
                               [--mode serial|parallel-warn|parallel-strict]
plan-auditor supervisor stop
plan-auditor supervisor status
plan-auditor task list | inspect <id>
plan-auditor plan verify <dir> | inspect <dir>
plan-auditor audit <dir>
plan-auditor evidence verify <dir>
plan-auditor doctor <dir>
plan-auditor agents list

The existing audit_check.py subcommands (validate, run, audit, status, snapshot, rollback) remain for Skill Mode.

Threat model

See docs/threat-model.md. Key points:

• Main AI is untrusted. The supervisor verifies independently.
• Same-user limitation. The supervisor cannot kernel-isolate from a malicious same-user agent. This is a documented limitation, not a guarantee. Use OS-level isolation for high-assurance cases.
• Fail-closed. Anything unknown resolves to FAIL or UNKNOWN, never PASS.
• Honest language. The project uses "tamper-evident", "fail-closed where supported", and "best-effort isolation" — never "unbreakable" or "immutable".

Guarantees

What's new in v2.0

• 14-layer supervisor architecture (L0-L14).
• Subsumption priority model.
• Plan sealing + monotonic verification.
• Multi-agent parallel support.
• Profiles (LIGHT/STANDARD/STRICT) and hardware tiers (TIER 1-4).
• Cross-archive evidence anchoring.
• 117 automated tests including 30 failure-injection / security scenarios.
• Full backward compatibility with Skill Mode.

License

MIT