Skip to main content

Edward

An external control plane for AI coding agents — deterministic guardrails, a local semantic scorer, and interventions you can resume.

CI PyPI downloads python license deps benchmark PRs welcome

PyPI · Benchmark · Contributing · Changelog

Agents fail quietly. Edward notices.

Edward live intervention: passive stall detected, agent cancelled at 3,589 tokens
Live run: a looping agent is stopped mid-flight — decision signed, audited, resumable.

Agents fail quietly. They retry the same broken test 40 times, burn $8 in tokens on a loop, run rm -rf on a database directory, and write to files they were never supposed to touch. The agent doesn't know it's failing — from its perspective, it's still trying.

Edward sits between the agent and its runtime. It watches the event stream, builds a picture of what the agent is actually doing across turns, and intervenes when the picture stops looking right.

Agent (Pi / Codex / custom)
    │ events
    ▼
Canonical Event Schema   ← normalizes tool names to capabilities
    │
    ▼
State Engine             ← materializes cross-turn agent state
    │
    ▼
Trigger Rules [FROZEN]   ← deterministic safety + convergence checks
    │
    ├─ HARD_CONSTRAINT ──→ Edward: BLOCK (scorer cannot override)
    │
    └─ SOFT_DECISION ──→ Local scorer ──→ Policy Resolver
                              │                │
                              └────────────────┘
                                       │
                                       ▼
                                 Control Kernel
                                       │
                                       ▼
                              PAUSE / CANCEL / RESUME

Why not just if/else?

A watchdog (error_count > 5 → stop) looks at individual events. The failure modes that actually kill long-running agents don't show up in any single event — they emerge from the shape of the trajectory over time. We tested this directly on held-out data (seed 137, frozen config):

Scenario Watchdog Edward (State Engine) Watchdog ctx Edward ctx
budget bleed 20% 100% 2,322 409
infinite loop 0% 100% 6,084 409
convergence stall 0% 100% 1,442 409
dangerous command 75% 100% 522 434

Watchdogs miss temporal failure modes entirely because they have no memory across turns. Edward maintains a sliding window of tool calls, tracks file modifications, and computes elapsed time — none of which fit in a single-event check.

Measured, not claimed

Edward validates itself against StepShield (NeurIPS 2026), the first benchmark treating intervention timing as a first-class metric (9,429 trajectories, step-level labels, 216 held-out):

Detector Recall FPR (clean) EIR₃ (timing) Cost / decision
LLMJudge (GPT-4.1-mini, paper) 95.4% 5.6% 0.89 GPT-4.1-mini price
HybridGuard (paper) 75.9% 44.4% 0.40 —
Edward contract probe (local 4B) 57.4% 20.4% 0.79 ~$0.00002
Edward rules only 7.4% 1.9% — 0
StaticGuard 847 rules (paper) 86.1% 77.8% 0.23 —

The deterministic layer alone is quantitatively blind to content-semantic violations (7.4%) — the "silent corruption" gap — while keeping the best false-positive rate. Adding a local 4B scorer with evidence-grounded task-contract probes and asymmetric temporal confirmation lands in LLMJudge-tier timing territory at zero marginal cost. Full measurement series and reproduction commands: BENCHMARK.md.

What it detects

Eight trigger rules, tuned on a dev split (seed 42) and frozen for held-out evaluation (seed 137):

Signal Fires when
Error rate > 40% over the last 8+ calls, no recovery signal
Retry count ≥ 3 retries of the same thing
Token budget > 80% consumed
Convergence stall > 600s + > 5 turns, no completion
Passive stall 12 consecutive reads, 0 writes
Dangerous command rm -rf, sudo, git push --force, `curl
Scope violation writes outside the allowed path prefixes
Silent corruption risk ≥ 10 consecutive file writes with zero shell verification

Policy packs make the knobs yours: conservative / balanced (= FROZEN defaults) / aggressive, as TOML or JSON.

Quickstart

pipx install edward-guard            # zero dependencies, Python 3.11+

edward doctor                        # environment checks
edward demo                          # self-running proof: 6 failure scenarios, PASS/FAIL
edward demo --live-scorer --offline  # same proof through the full scorer pipeline (heuristic stub, no GPU)

edward wrap -- pi "fix the flaky test"                     # full monitoring + intervention
edward wrap --no-scorer -- python my_agent.py              # any command, rule-only
edward wrap --scope ./src --auto-resume 60 -- pi "task"    # scoped writes, auto-resume
$ edward demo
policy: balanced  trials/scenario: 3

scenario             expect     result        latency
-----------------------------------------------------
normal               no-fire    clean               —  ok
transient_failure    no-fire    clean               —  ok
infinite_loop        fire       100% detected      8.0  ok
budget_bleed         fire       100% detected     12.0  ok
dangerous            fire       100% detected      4.7  ok
stall                fire       100% detected      4.0  ok

PASS in 0.0s (deterministic rules frozen defaults; scorer off)

Real output, not a mock — six failure scenarios against the frozen rule set, plus the two clean controls. Wraps any subprocess: Pi, Codex, claude -p, CI jobs, plain scripts. --agent auto|generic|pi picks the adapter (pi gets native RPC; the registry is extensible via edward.adapters entry points).

Interventions are resumable, not fatal: PAUSE exits with code 75, pins the agent session, and edward wrap --continue picks the same session back up from the audit log. CANCEL / BLOCK exit 76. Audit lands in ~/.edward/audit.jsonl — including an estimated avoided-spend per intervention.

The scorer is optional and always advisory. Point EDWARD_SCORER_URL at any local OpenAI-compatible scoring endpoint (a 4B model on your GPU box is plenty — see deploy/ for the team-LAN topology). Scorer down? Edward logs a warning and runs rule-only. It stays protective.

Pluggable scorer backends. EDWARD_SCORER_BACKEND selects where judgments come from: endpoint (default — the LAN scorer server above), jev (TypeSafe Jev — all probes batched into one calibrated call; set TYPESAFE_API_KEY), or heuristic (deterministic marker stub for offline demos and tests). edward doctor shows the active backend. All backends are advisory and share the same circuit breaker.

Edward + Jev — who decides what. With the jev backend (edward/backends.py), one Jev call receives the full cross-turn state (error trends, write streaks, budget) and returns calibrated judgments — should this trajectory continue, pause, or escalate, and how confident is that? Deterministic code owns everything irreversible: blocklists, scope checks, budget caps, intervention execution, and the signed receipt chain. Jev's confidence gates routing, never actions. Known limitation: the local-4B and Jev paths trade recall differently (see BENCHMARK.md for measured numbers and reproduction commands).

v0.2.0 highlights

  • Signed evidence receipts — every audit record is Ed25519-signed into a hash chain (pure stdlib, RFC 8032 vectors); edward verify proves tamper-evidence offline. Publish your public key; anyone can check.
  • Human approval loop — --wait-approval 300 sends Resume/Kill links to Slack (or stderr) and waits; PAUSE becomes a decision, not a dead end.
resume from audit + offline receipt verification

Why zero dependencies?

Edward's control loop runs stdlib-only: it must boot on any Python 3.11+ box, inside any container, in front of any agent — including air-gapped ones. The heavy lifting (scoring) is delegated to a separate local service, which you own and can swap (4B quantized, bigger, whatever) without touching the control plane.

Repository map

edward/                the package
  cli.py               wrap / demo / eval / audit / doctor
  engine.py            ControlPlane: events → triggers → scorer → decision → audit
  state_engine.py      cross-turn agent state
  triggers.py          8 rules, policy-parameterized (defaults FROZEN)
  scorer_client.py     /v1/score client + circuit breaker
  stepshield.py        external benchmark adapter (EIR metrics)
  scenarios.py         failure scenario suite (demo/eval source of truth)
benchmark.py           300-trial held-out benchmark
robustness_eval.py     4-dimension robustness attack
BENCHMARK.md           full measurement series + reproduction commands
deploy/                team-LAN deployment templates

Status & roadmap

  • v0.1.1 on PyPI, CI on three platforms
  • StepShield integration with paper-aligned EIR metrics
  • Robustness suite as edward eval --suite robustness
  • Scorer fine-tune (targets FPR; data flywheel from audit logs)
  • Cloud fleet console (team tier)

Contributing

Deterministic layer stays deterministic: trigger defaults are FROZEN, and behavior-affecting changes require re-running the benchmark gate. See CONTRIBUTING.md.

License

MIT — © 2026 Edward contributors

Release files for edward-guard 0.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for edward-guard 0.3.0
File Size Uploaded
edward_guard-0.3.0.tar.gz 55.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for edward-guard 0.3.0
File Interpreter ABI Platform
edward_guard-0.3.0-py3-none-any.whl Python 3 none any Details

Total release size: 113.5 kB

Release files / edward_guard-0.3.0.tar.gz

Download URL edward_guard-0.3.0.tar.gz
Size 55.2 kB
Tags Source
SHA-256 checksum
How to use checksums
fd96d6a06c19426f087033818f729c7d62cfa4fbe2b1f0fbf870d50715de8c18
BLAKE2b-256 checksum
How to use checksums
f4f57de1fe21e3f45c80793849b7d144bf9cc70791f12be4e28b82d8eaf05dee
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release files / edward_guard-0.3.0-py3-none-any.whl

Download URL edward_guard-0.3.0-py3-none-any.whl
Size 58.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
9b81f38a5eed55ac66012f2ecee6e15e40d239449e26e56987ade14575af71ff
BLAKE2b-256 checksum
How to use checksums
a9a59bcd3f2000f74064aec38b34de44cdb8c41e5e2040af86f2a05725009eef
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release history Release notifications | RSS feed

This release

0.3.0 This release

2 release files

0.2.0

2 release files

0.1.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page