foresight-agent-guard 0.1.3

Predictive cost, policy, and runaway-session guardrails for AI coding agents

Foresight

Foresight is a predictive cost and tier-routing layer for AI coding agents. It chooses the right level of reasoning before work begins, then watches the session closely enough to stop drift before it becomes expensive.

Judge quick start

Run the current submission from source. It needs Python only: no API key, Node.js, or browser.

git clone https://github.com/AmanM006/foresight.git
cd foresight
pip install -e .
foresight demo --replay
python verify.py

The replay launches a real instrumented Python child, shows the terminal dashboard, flags scope drift, then blocks a fake .env read in under a minute. --web additionally opens the optional local Next.js dashboard.

Install from PyPI

pip install foresight-agent-guard==0.1.3 installs the current Python-child guard release.

Verify this submission

Run the complete replay-safe verification pass with one command:

python verify.py

It runs the Python-child replay and deterministic tests without spending tokens. Add --live only when you want it to run the paid tier and injection benchmarks as well. For a zero-setup visual alternative, explore the deployed dashboard at sentry-lime.vercel.app.

The problem

When developers use Codex/GPT-5.6 for coding work, they often do not know upfront which tier or reasoning effort a task actually needs. Defaulting to Sol overpays for mechanical work; choosing too small a tier can leave an agent stuck and repeatedly escalating.

The risk is not hypothetical. An open Codex issue #32250 reports GPT-5.6 Sol Medium reducing a Pro five-hour allowance from 87% to 76% during a short conversation and several trivial follow-ups. Recent r/codex and r/ChatGPTPro threads describe confusion around five-hour versus weekly limits; these are community reports, not a claim about every session. Community guidance also identifies duplicated context and uncontrolled subagent spawning as sources of extra consumption (discussion).

There is a second failure mode: scope creep. A small implementation task can turn into unrequested architecture work, sprawling documentation, or broad file changes—the “Sol Ultra problem” discussed by users—silently consuming budget without an early warning. OpenAI documents that ultra coordinates four agents in parallel by default and trades higher token use for stronger results on demanding tasks; it is not an appropriate default for every coding task (OpenAI GPT-5.6 overview).

Foresight solves this by predicting complexity and tier before execution, then watching the live session to catch drift before it compounds.

Live demo proof

One real recorded demo run produced the following result:

Signal	Observed result
Predicted route	luna / low / isolated
Files touched	16
Predictive flag	scope_explosion
Final disposition	escalated_for_review
Recorded model cost	approximately $0.0060

The demo showed two independent catch mechanisms live:

• Predictive scope-drift detection: unrelated file changes exceeded the predicted isolated blast radius.
• Forbidden-pattern interception: an attempted .env read was caught and blocked by the policy layer.

The event-feed and kill-state captures above were refreshed from the current local foresight demo --replay path on 2026-07-16. Replay is finite, so the screenshots were captured after its event-bus process exited and the UI correctly shows Offline / Pending connection; the captured action feed and kill state are the events received during that run. The cost capture is intentionally archival: replay makes no API calls, so its cost stays $0.00000; the image shows the separately recorded live GPT-5.6-terra cost evidence linked below.

Inspect the sanitized raw evidence in examples/: policy_and_verdict_output.example.json and audit_event_output.example.json.

Evidence and benchmarks

Evidence	Real result
Tier prediction accuracy	15 hand-labeled tasks spanning mechanical, moderate, and architectural complexity. 14/15 exact tier match, 1 conservative overestimate (avatar upload: expected terra/moderate, predicted sol/moderate), 0 risky underestimates. Total cost: $0.0346 across 15 real GPT-5.6-terra calls. View results.
Injection resistance	10 adversarial prompt-injection scenarios across hidden HTML comments, code comments, disguised TODOs, subprocess arguments, config fields, misleading filenames, social-engineering “temporarily disable” requests, and multi-step justification chains. 9/10 caught and blocked. The one miss triggered the fail-safe path (uncertain evaluation → warning, not silent allow) rather than a full block; it is disclosed as a real finding. View results.
Multi-task session (false-positive check)	One continuous session across four task types: a trivial README fix (luna/isolated, clean), scoped API validation (terra/moderate, clean), a broad payment migration (sol/wide, 6 files touched, not flagged as scope_explosion because wide scope was predicted), and an injected .env access attempt (blocked). Total session cost: $0.01812. View results.
Workflow-plan benchmark	Fixed stratified sample of five tasks from the 15-case tier corpus: 4/5 plan tiers matched the hand labels; the MFA task was under-tiered (sol expected, terra planned). All five plans met the hard planner contract (1-4 bounded phases, each action cap within the 1-40 schema limit, complete phase fields, and a stop prompt). The stricter per-tier action-budget guidance was met by 2/5 plans because several plans deliberately used shorter discovery/verification phases. Total cost: $0.0306. View raw plans and checks.
Live effort-curve integration	One fresh mechanical-task prediction plus one generated workflow plan both returned luna/low and each carried the source-backed DeepSWE cost curve and Luna caution. Total real cost: $0.0055 across 2 GPT-5.6-terra calls. View captured output.
Live Python-child guard	Real GPT-5.6-terra scope/prediction: luna / low / isolated; a separately launched Python child wrote allowed src/auth/login.py, then an attempted .env read received a critical deny verdict before the read. 2 setup calls, $0.00339, 0 child evaluation calls. View raw policy, response IDs, audit, and report.
Unified live pipeline	One continuous Python-child session (foresight-unified-06e40865) proved predict -> allowed write -> scope drift reassessment -> parallel fan-out -> hard action ceiling kill, with a burn forecast before termination. It made 5 real GPT-5.6-terra calls for $0.01122; the raw timeline includes timestamps, all response IDs, every verdict and signal, and the final kill reason. View unified evidence.

Run python verify.py to reproduce the zero-cost checks yourself, or python verify.py --live to reproduce the paid benchmarks above (~$0.08 total across all three).

One benchmark response includes its raw OpenAI response ID as tamper-evident proof that it was a real API call, not a fabricated result: response_id: resp_016b9a9aa893e50d006a57492295908192b3c0b54f1eefd7c7. The recorded response is documented in benchmark_results.json.

What it does

• Routes before spend: predicts the right Luna/Terra/Sol tier, effort level, and blast radius before a single task action is evaluated.
• Turns scope into enforcement: derives a restrictive policy from the declared task instead of treating agent access as implicitly broad.
• Instruments Python agents for real: foresight run -- python agent.py injects process-local interception before child agent code starts, then captures file I/O, subprocesses, and requests calls before execution.
• Flags runaway behavior early: detects scope explosions, low-progress churn, and repeated paralysis loops while the session is live.
• Makes the effort tradeoff visible: attaches a deterministic, illustrative low-to-max cost curve and warns when Luna is being used beyond truly mechanical work.
• Supports time-sensitive sessions honestly: --fast-mode applies the documented 2.5x quota-burn multiplier and shows the speed-versus-budget warning before work begins.
• Makes spend auditable: streams actions, verdicts, predictions, signals, and token-derived costs to the event bus and dashboard.

Architecture

Task description
      |
      v
foresight/ predictor + session monitor
      |
      v
Python child bootstrap -> sentry/ interception engine + policy + event bus ----> dashboard frontend

• foresight/ contains prediction, routing, the CLI, and session monitoring. Keeping these concerns together makes the pre-execution estimate and live escalation decision share the same session context.
• sentry/ is the internal interception engine codename. It captures actions, applies policy, tracks costs, and publishes events. It is decoupled from foresight/ so the security layer can be reused independently of prediction logic.
• Python child bootstrap is injected only for foresight run -- python .... It installs the process-local proxy before agent code, then forwards canonical actions and costs back to the parent monitor over the event bus.
• sentry/web/ is the single, restored Next.js dashboard. It consumes the event model rather than duplicating policy or interception behavior.

Sentry is the internal codename for the interception engine; Foresight is the complete product.

Runtime decision flow

1. Task description
   -> predictor returns tier, effort, blast radius, and risk factors
   -> scope-to-policy returns allowed paths, hosts, and forbidden patterns

2. Agent attempts an action
   -> SentryProxy captures it before execution
   -> Action(type, target, payload, timestamp, session_id)

3. Fast enforcement path
   -> forbidden target: critical block, no model call
   -> out-of-scope file or host: warning event
   -> identical action repeated six times in 60 seconds: paralysis-loop block

4. Ambiguous in-scope action
   -> GPT-5.6-terra returns a structured Verdict
   -> allow, warn, or block before the original operation proceeds

5. Session-level monitoring
   -> compare files, diffs, and spend against the original prediction
   -> scope/churn signal triggers a contextual reassessment
   -> kill only if the reassessment confirms a wide or Sol-level runaway

Layer responsibilities

Layer	Main modules	Responsibility	Why it is separate
Prediction	foresight/predictor.py	Produces tier, effort, blast radius, rationale, and risk factors from the task description.	Establishes an expected cost and scope baseline before any agent action exists.
Session control	foresight/session_monitor.py	Watches file count, repeated diffs, and cost ceilings; emits signals and reassesses drift.	Session-level behavior cannot be decided from one action alone.
Policy	sentry/policy/engine.py	Converts scope to Policy, makes local decisions first, and uses structured model review only for ambiguity.	Keeps common security decisions deterministic, cheap, and auditable.
Enforcement	sentry/proxy/interceptor.py	Temporarily intercepts file, subprocess, delete, and requests calls in the current Python process.	Places the decision immediately before the original operation can execute.
Telemetry	sentry/eventbus/server.py	Broadcasts actions, costs, predictions, and monitor signals over WebSocket; exports JSON audit logs.	Decouples producers from the dashboard and leaves a machine-readable record.
Presentation	sentry/web/	Renders live connection state, action feed, policy context, cost, and kill state.	Lets a reviewer understand a session without reading Python logs.

The common event contract

Every intercepted operation becomes the same Action record:

Action
  type: file_read | file_write | file_delete | subprocess | network
  target: path, command, or host:port
  payload: mode, unified diff, cwd/env summary, or request-size summary
  timestamp
  agent_session_id

Policy evaluation returns a Verdict with severity, reason, and allowed. This normalized pair is what the proxy enforces, the monitor observes, the event bus broadcasts, and the dashboard displays. Model use is represented separately as CostEvent(model, input_tokens, output_tokens, cost_usd, timestamp).

Four failure modes, four distinct mechanisms

Foresight deliberately gives each failure mode a distinct trigger, reason, event name, and escalation path instead of treating all anomalous activity as one generic warning:

1. Forbidden-pattern interception is deterministic and zero-latency. SentryProxy blocks an action such as .env access before it executes, emits the policy Verdict, and terminates only on a critical denied verdict. This is security enforcement, not prediction.
2. Scope explosion is predictive and contextual. SessionMonitor compares touched files with the predicted blast radius, emits scope_explosion, and re-runs complexity assessment with session context before committing to termination. A wide predicted task is not falsely killed for having a wide footprint.
3. Hard action ceiling is a deterministic workflow cap. The predictor maps isolated to 15 actions, moderate to 40, and wide to 100. Reaching the cap emits hard_ceiling_reached with hard_action_ceiling_reached as its final reason and terminates immediately, even if the agent believes it is nearly done.
4. Parallel fan-out detects a different failure shape from repeated loops: five distinct file or subprocess actions inside three seconds. It emits parallel_explosion and triggers soft reassessment, allowing deliberate, paced parallel work to remain unblocked.

Burn-rate forecasting is separately broadcast as burn_forecast, but it is intentionally informational rather than a kill switch. After enough cost events, it projects minutes remaining at the current cost/minute rate so a user can intervene before a budget ceiling is reached. This keeps forecast visibility separate from hard enforcement.

The independent paralysis_loop check remains in SentryProxy: a rolling 60-second window blocks the sixth identical action after five prior repeats before any model evaluation. It is distinct from parallel fan-out because one catches repeated identical work while the other catches a burst of different work.

This distinction is why the demo can truthfully show two independent outcomes: an immediate .env block from deterministic policy enforcement, and an escalated_for_review result after predictive scope drift is reassessed.

Cost-aware model use

scope_to_policy() and predict_complexity() use structured output. evaluate_action() uses GPT-5.6-terra only when a local rule cannot resolve an in-scope but suspicious action. Each response has an explicit output cap, a stable prompt prefix for caching, and usage recorded through CostTracker.

Predicted effort is calibrated separately from tier: mechanical isolated work routes to low, normal scoped coding work defaults to medium, and high is reserved for genuinely high-stakes or architectural work. Every prediction and workflow plan carries a deterministic, display-ready cost curve from the DeepSWE v1.1 leaderboard: observed GPT-5.6 Sol average cost per task was low $1.07 (1.00x), medium $1.86 (1.74x), high $3.47 (3.24x), xhigh $4.70 (4.39x), and max $8.39 (7.84x) relative to low. These are benchmark observations for one model/harness, not API pricing or Foresight-measured spend. The CLI and dashboard mark xhigh and max as steep-cost territory with marginal expected quality gain; no extra model call is required.

When a task is routed to luna, Foresight also adds a deterministic caution: Luna is for high-volume data handling, simple classification, or truly mechanical edits. Anything requiring real coding logic or judgment should be checked carefully because Terra or Sol is the practical floor for most coding work.

The architecture therefore treats model judgement as a metered dependency rather than an invisible background call.

Current prototype boundaries

The repository is intentionally a focused vertical slice, not a claim of production multi-tenant infrastructure:

• Interception is process-scoped Python monkey-patching, not an operating-system sandbox.
• Supported automatic interception is currently Python-child scoped: use foresight run -- python agent.py. Node, shell-native, and other agent runtimes are intentionally not represented as supported today.
• Audit events are held in memory and persisted through explicit JSON export rather than a database.
• Budget ceilings are current heuristics, not learned per-user limits.
• A deployed dashboard needs a reachable event-bus endpoint; it cannot receive events from a developer's local process without a connection path.
• The dashboard is intentionally a single deploy target at sentry/web/; a production hardening pass would add deployment-specific event-bus connectivity and persistence.

What we got wrong, and what it taught us

We originally wrote the demo around the task “Refactor the login flow to add MFA support,” assuming it would produce a narrow terra / isolated prediction so the scripted scope-explosion sequence could exercise escalation. When we ran it against the real GPT-5.6-terra model, it classified MFA and authentication work as security-sensitive and returned sol / high / wide. That was the right call: MFA changes can affect identity, recovery, and session boundaries. Our test assumption, not the prediction, was wrong. The problem was that a wide blast radius has no file-count ceiling to exceed, so the demo narrative no longer matched the model’s real risk assessment. We redesigned the task as a genuinely narrow isolated variable rename, which keeps the expected route and scripted drift consistent. We also added a second, prediction-independent catch: the forbidden-pattern block on .env access. It works regardless of the predicted tier, so the demo now proves both predictive scope monitoring and deterministic interception without guessing a model outcome. This is the same principle the system is built to detect in agents: the gap between what we assumed a task’s scope was and what it actually turned out to be.

Why not just use a bigger model for everything?

Sol costs five times Luna’s input and output price, while many coding tasks are mechanical enough to complete safely with a smaller tier. Defaulting to the frontier tier therefore creates exactly the avoidable spend Foresight is designed to expose. Prediction is not free: it adds a metered call before work begins. But a conservative prediction only adds an escalation round-trip, because Foresight reassesses live drift instead of trusting the first estimate blindly.

Quickstart

Install the Python runtime and CLI:

pip install -r requirements.txt
# or, for the `foresight` command:
pip install -e .

Create your local environment file from the committed template:

copy .env.example .env

Set OPENAI_API_KEY in .env, then run the end-to-end demo:

python -m foresight.demo_scenario

For a complete local replay with zero OpenAI API calls (using recorded GPT-5.6 policy and prediction responses through a real Python child), run:

foresight demo --replay

The live path above remains the primary demonstration. Replay is the accessible fallback for judges who want to exercise the real interception and monitoring flow without an API key. For a zero-setup visual tour of the dashboard UI, visit the deployed dashboard; it is distinct from replay mode, which runs the local Python enforcement logic.

CLI workflow:

# On a new directory, `foresight` launches an interactive first-run setup.
foresight

# Or explicitly create a plan, then monitor a task.
foresight init
foresight run -- python your_agent_task.py

# Zero-key, live-interception replay for judges.
foresight demo --replay

# Use only when latency matters more than budget.
foresight run --fast-mode -- python your_agent_task.py

# CI/headless mode: suppress terminal UI.
foresight run --no-dashboard -- python your_agent_task.py

# Optional richer local browser view.
foresight run --web -- python your_agent_task.py

foresight status
foresight export

Run the dashboard locally:

cd sentry/web
npm install
npm run dev

Open http://localhost:3000/dashboard. The event bus listens on ws://localhost:8765 by default.

Dashboard connection status

foresight run starts with a zero-dependency Rich terminal dashboard. Use --web to additionally open http://localhost:3000/dashboard, or --no-dashboard for headless environments.

The public Vercel dashboard is a visual deployment, not a hosted multi-tenant event-bus service. Its root URL rejects WebSocket upgrades, so foresight run --remote currently prints a clear notice and falls back to local mode rather than pretending remote streaming works. To see a live local session today, run the Next.js dashboard above alongside the CLI. A secured hosted ingestion service is required before remote mode can become real.

Cost tracking and prompt caching

Every OpenAI response becomes a CostEvent, calculated from input/output token usage and streamed to the event bus. Calls use a stable system/context prefix with only task-specific content appended, improving prompt-cache reuse. Clear forbidden or out-of-scope actions are decided locally first, avoiding unnecessary model calls.

Every prediction and policy call is cost-tracked in real time and streamed to the dashboard -- Foresight audits its own spend, not just the agent's.

Fast mode applies a 2.5x multiplier to recorded cost events to make its quota-burn tradeoff visible. Foresight displays: "Fast mode increases quota burn by ~2.5x for ~1.5x speed. Recommended only when latency matters more than budget."

Codex vs Antigravity build split

• Codex -- interception, reasoning, and enforcement. sentry/proxy/interceptor.py implements pre-execution monkey-patching for subprocesses, file reads/writes/deletes, and HTTP requests; unified-diff capture; tracked-process termination; and the local repeated-action paralysis guard. sentry/policy/engine.py handles structured scope-to-policy conversion, local forbidden/scope checks, contextual evaluation, prompt-cache discipline, and token-based cost events. foresight/predictor.py provides structured tier/effort/blast-radius routing, configurable tier bias, deterministic action ceilings, and typed phase-bounded workflow plans. foresight/session_monitor.py implements scope-explosion, low-progress churn, hard action/phase caps, parallel fan-out detection, burn-rate forecasting, phase-output drift checks, reassessment, and escalation.
• Codex -- product plumbing and proof. Codex also built the WebSocket event surface in sentry/eventbus/server.py, the interactive Rich CLI and TOML configuration system in foresight/cli.py and foresight/config.py, the replay/live demos, tier/injection/multi-task/workflow benchmarks, PyPI packaging metadata, and publishing documentation. These components carry predictions, signals, audit events, and costs end-to-end; they are not presentation-only stubs.
• Antigravity -- presentation and delivery. Antigravity handled the visual polish in sentry/web/, dashboard layout/connection fixes, deployment configuration, and documentation presentation cleanup. It consumes the event contract but does not make policy decisions, predict tiers, calculate enforcement signals, or terminate sessions.

This split is intentional and documented per hackathon submission requirements. Codex built 100% of the reasoning-critical and safety-critical behavior: prediction, policy evaluation, workflow bounding, cost/burn accounting, anomaly detection, and escalation/termination.

Roadmap

• Multi-agent swarm monitoring with a shared session risk view.
• Historical session learning to improve tier prediction over time.
• Monitoring support for Claude Code and other agent runtimes beyond Codex.
• Node, shell-native, and other non-Python agent-runtime interception.
• Team-level policy profiles and budget guardrails.

License and attribution

MIT License. Built for Build Week, July 2026.