Skip to main content

PlannerCritic Engine

License: MIT Python 3.11+ PyPI Ruff Type checked Coverage Contributor Covenant OpenSSF Field Test

Hierarchical task planning with an independent LLM critic. A planner decomposes a goal into a structured plan; a critic audits every subtask; the plan is revised until approval — or escalated to a human.

[!NOTE] Status: v0.1.0 released · PyPI · pip install planner-critic License: MIT


Why

Planning is the weakest part of agent systems. Most agents act too early, skip hard subproblems, and start executing a plan that was never reviewed. A single-pass decomposition embeds silent assumptions, and the first sign of trouble arrives mid-execution — after state has already diverged.

Single-pass planning fails silently on multi-step goals, and a model "reviewing" its own plan is agreement with extra steps. There is no draft to review, no independent reviewer to catch the gap, and no structured escalation.

PlannerCritic Engine closes this gap by treating planning as a first-class, productized artifact rather than a hidden chain-of-thought side effect.


What It Is

The Draft → Critique → Revise → Escalate Loop

 Goal + Constraints → PLANNER → typed plan → CRITIC → findings
                         ↑  │                        │
                         │  └────── revise ←──────────┘
                         │                             │
                         └──────── approved plan ──────┘
                                      │
                                   ESCALATE (if no convergence)
  • Draft — a planner LLM decomposes a goal into a structured, typed plan: tasks, dependencies, ordering, verification steps, and rollback points.
  • Critique — a separate critic LLM audits every subtask against six heuristic families: feasibility, risk, missing steps, unsafe sequencing, unverified dependencies, weak rollback — producing severity-graded findings.
  • Revise — the planner revises in response, in a bounded loop with a revision budget and convergence detection, preserving draft history.
  • Escalate — if the loop cannot converge, a human gets a minimal, precise question about exactly what is blocking approval.

The plan is a persisted, versioned artifact — you can diff revisions, see which critiques drove which changes, and trace whether a failed run was a planning failure or an execution failure.

Key Features

Feature Description
Risk tolerance balanced (findings are advisory warnings) or strict (zero tolerance, fail-closed)
Deterministic gates 7 injection-immune gates — ordering, branch-sanity, rollback, verification, preconditions, branch-tasks, high-risk completeness
Escalation management Human-in-the-loop with override, patch, and restart decisions
Convergence detection Early termination when the planner stops making progress — saves LLM calls
Provider registry Pluggable LLM providers (OpenRouter, OpenAI, oMLX, Ollama) via TOML config
StructuredEnforcer Retry mechanism for LLM JSON output — fail-closed after 3 retries
Plan versioning Every revision is a persisted artifact with diff support

What It Is Not

  • It does not execute the plan — an existing runner consumes the approved plan.
  • It does not guarantee plan correctness — it reduces risk, it cannot eliminate it.
  • It does not replace execution engines or agent frameworks (LangGraph, CrewAI, etc.).

Quick Start

pip install planner-critic
plancritic quickstart  # creates a demo goal and runs the loop
plancritic demo        # runs the full demo scenario

Requires Python 3.11+ and an LLM provider (OpenRouter, OpenAI, or local model).


CLI

plancritic plan <goal.json>              # Plan a goal
plancritic critique <plan.json>          # Critique a plan
plancritic field-test run --goals <dir>   # Run field test
plancritic demo                          # Run demo scenario
plancritic quickstart                    # Quickstart demo
plancritic migrate <old> <new>           # Migrate config
plancritic serve                         # Start HTTP server

See API Reference for full CLI docs, HTTP endpoints, and MCP tools.


Field Test

157 goals across 35 domains, all run against a real LLM (gpt-4o-mini via OpenRouter):

Metric Result
Balanced goals approved 71/71 (100%)
Strict goals escalated 81/81 (100%)
Adversarial goals escalated 8/8 (100%)
True failures 0
Deterministic gate passes 156/157 (99%)
Scorecard A (post-amendment) PASS
Scorecard B (pass* semantics) 100%

Full results: field-test-results-0.1.0.md


Documentation

Doc Path Contents
Field Test Results v0.1.0 results BLUF, conclusions, per-goal data, scorecards, blocker analysis
Field Test Plan plan 156-goal corpus, 35 capabilities, invariant assertions
Architecture v0.1.0 architecture Component diagram, module map, data flow
API Reference api CLI cheat-sheet, HTTP endpoints, MCP tools
Design Decisions decisions DD-01..N decision records
Demo Scenario demo End-to-end walkthrough
WBS Index wbs Milestone overview, dependency graph

Project Layout

planner-critic-engine/
├── docs/                    Documentation
│   ├── architecture/          System architecture and spec
│   ├── design/                PRD, design spec, design decisions
│   ├── field-test/            Field test plan + results (157 goals, 35 domains)
│   ├── reference/             API reference, quickstart
│   └── wbs/                   Work breakdown structure (M1–M10)
├── src/planner_critic/       Engine source
│   ├── cli/                    CLI commands
│   ├── critique/               LLM critic with severity guardrail
│   ├── gates/                  7 deterministic gates
│   ├── llm/                    Provider registry, transport, logging
│   ├── loop/                   Plan revision loop, convergence detection
│   └── server/                 HTTP server
├── tests/                    Test suite
├── .github/                  Issue templates, PR template, CI workflows
├── CHANGELOG.md               Release history
├── CONTRIBUTING.md            How to contribute
├── SECURITY.md                Security policy + OWASP + OpenSSF
└── pyproject.toml             Package metadata

Known Gaps (v0.2.0)

  • Planner capability gap — 132 concrete blockers across 63 strict goals. A deterministic precondition closer would eliminate 48%.
  • CLI, HTTP, adapter surfaces — partial coverage
  • Multi-model sweeps — only gpt-4o-mini tested
  • Finding quality audit — not yet measured
  • Executor usability — not yet audited

See CHANGELOG.md for full details.


License

MIT — see LICENSE.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

planner_critic-0.1.0.tar.gz (1.4 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

planner_critic-0.1.0-py3-none-any.whl (250.1 kB view details)

Uploaded Python 3

File details

Details for the file planner_critic-0.1.0.tar.gz.

File metadata

  • Download URL: planner_critic-0.1.0.tar.gz
  • Upload date:
  • Size: 1.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.5

File hashes

Hashes for planner_critic-0.1.0.tar.gz
Algorithm Hash digest
SHA256 cf3a9ac8f5ae8a8404037e821fd47760f3a157de483ced87022e18615ff34c69
MD5 93600c85a8c17e242f1ad667748af470
BLAKE2b-256 f2671e2e48405da111226b11ee7178569ec3827a72e349e0f4849e7c3f11a71b

See more details on using hashes here.

File details

Details for the file planner_critic-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: planner_critic-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 250.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.5

File hashes

Hashes for planner_critic-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 c8c8d20f6bcb54fd5ff77393fbdccb701239df73126a4b197c37541aade8a808
MD5 55c3a1f1d00a300cb8e5391d49706f51
BLAKE2b-256 50790e0d61eb24f4814ae350f23062b75e3bfc20a953ad5420fb1e728dae5f46

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page