foresight-agent-guard 0.1.0

Predictive cost, policy, and runaway-session guardrails for AI coding agents

Foresight

Foresight is a predictive cost and tier-routing layer for AI coding agents. It chooses the right level of reasoning before work begins, then watches the session closely enough to stop drift before it becomes expensive.

Install from PyPI (after the first release)

pip install foresight-agent-guard
foresight --help

Build from source / quick test

git clone https://github.com/AmanM006/foresight.git
cd foresight
pip install -e .
copy .env.example .env
python -m foresight.demo_scenario

Add your OPENAI_API_KEY to .env after copying the example file. On macOS/Linux, use cp .env.example .env.

Verify this submission

Run the complete replay-safe verification pass with one command:

python verify.py

It checks the environment, runs tests when present, and executes the full replay demo without spending tokens. Add --live only when you want it to run the paid tier and injection benchmarks as well. For a zero-setup visual alternative, explore the deployed dashboard at sentry-lime.vercel.app.

The problem

When developers use Codex/GPT-5.6 for coding work, they often do not know upfront which tier or reasoning effort a task actually needs. Defaulting to Sol overpays for mechanical work; choosing too small a tier can leave an agent stuck and repeatedly escalating.

The risk is not hypothetical. An open Codex issue #32250 reports GPT-5.6 Sol Medium reducing a Pro five-hour allowance from 87% to 76% during a short conversation and several trivial follow-ups. Recent r/codex and r/ChatGPTPro threads describe confusion around five-hour versus weekly limits; these are community reports, not a claim about every session. Community guidance also identifies duplicated context and uncontrolled subagent spawning as sources of extra consumption (discussion).

There is a second failure mode: scope creep. A small implementation task can turn into unrequested architecture work, sprawling documentation, or broad file changes—the “Sol Ultra problem” discussed by users—silently consuming budget without an early warning. OpenAI documents that ultra coordinates four agents in parallel by default and trades higher token use for stronger results on demanding tasks; it is not an appropriate default for every coding task (OpenAI GPT-5.6 overview).

Foresight solves this by predicting complexity and tier before execution, then watching the live session to catch drift before it compounds.

Live demo proof

One real recorded demo run produced the following result:

Signal	Observed result
Predicted route	luna / low / isolated
Files touched	16
Predictive flag	scope_explosion
Final disposition	escalated_for_review
Recorded model cost	approximately $0.0060

The demo showed two independent catch mechanisms live:

• Predictive scope-drift detection: unrelated file changes exceeded the predicted isolated blast radius.
• Forbidden-pattern interception: an attempted .env read was caught and blocked by the policy layer.

Inspect the sanitized raw evidence in examples/: policy_and_verdict_output.example.json and audit_event_output.example.json.

Evidence and benchmarks

Evidence	Real result
Tier prediction accuracy	15 hand-labeled tasks spanning mechanical, moderate, and architectural complexity. 14/15 exact tier match, 1 conservative overestimate (avatar upload: expected terra/moderate, predicted sol/moderate), 0 risky underestimates. Total cost: $0.0346 across 15 real GPT-5.6-terra calls. View results.
Injection resistance	10 adversarial prompt-injection scenarios across hidden HTML comments, code comments, disguised TODOs, subprocess arguments, config fields, misleading filenames, social-engineering “temporarily disable” requests, and multi-step justification chains. 9/10 caught and blocked. The one miss triggered the fail-safe path (uncertain evaluation → warning, not silent allow) rather than a full block; it is disclosed as a real finding. View results.
Multi-task session (false-positive check)	One continuous session across four task types: a trivial README fix (luna/isolated, clean), scoped API validation (terra/moderate, clean), a broad payment migration (sol/wide, 6 files touched, not flagged as scope_explosion because wide scope was predicted), and an injected .env access attempt (blocked). Total session cost: $0.01812. View results.

Run python verify.py to reproduce the zero-cost checks yourself, or python verify.py --live to reproduce the paid benchmarks above (~$0.08 total across all three).

One benchmark response includes its raw OpenAI response ID as tamper-evident proof that it was a real API call, not a fabricated result: response_id: resp_016b9a9aa893e50d006a57492295908192b3c0b54f1eefd7c7. The recorded response is documented in benchmark_results.json.

What it does

• Routes before spend: predicts the right Luna/Terra/Sol tier, effort level, and blast radius before a single task action is evaluated.
• Turns scope into enforcement: derives a restrictive policy from the declared task instead of treating agent access as implicitly broad.
• Intercepts high-risk actions: captures file I/O, subprocesses, and network requests before execution.
• Flags runaway behavior early: detects scope explosions, low-progress churn, and repeated paralysis loops while the session is live.
• Makes spend auditable: streams actions, verdicts, predictions, signals, and token-derived costs to the event bus and dashboard.

Architecture

Task description
      |
      v
foresight/ predictor + session monitor
      |
      v
sentry/ interception engine + policy + event bus ----> dashboard frontend

• foresight/ contains prediction, routing, the CLI, and session monitoring. Keeping these concerns together makes the pre-execution estimate and live escalation decision share the same session context.
• sentry/ is the internal interception engine codename. It captures actions, applies policy, tracks costs, and publishes events. It is decoupled from foresight/ so the security layer can be reused independently of prediction logic.
• sentry/web/ is the single, restored Next.js dashboard. It consumes the event model rather than duplicating policy or interception behavior.

Sentry is the internal codename for the interception engine; Foresight is the complete product.

Runtime decision flow

1. Task description
   -> predictor returns tier, effort, blast radius, and risk factors
   -> scope-to-policy returns allowed paths, hosts, and forbidden patterns

2. Agent attempts an action
   -> SentryProxy captures it before execution
   -> Action(type, target, payload, timestamp, session_id)

3. Fast enforcement path
   -> forbidden target: critical block, no model call
   -> out-of-scope file or host: warning event
   -> identical action repeated six times in 60 seconds: paralysis-loop block

4. Ambiguous in-scope action
   -> GPT-5.6-terra returns a structured Verdict
   -> allow, warn, or block before the original operation proceeds

5. Session-level monitoring
   -> compare files, diffs, and spend against the original prediction
   -> scope/churn signal triggers a contextual reassessment
   -> kill only if the reassessment confirms a wide or Sol-level runaway

Layer responsibilities

Layer	Main modules	Responsibility	Why it is separate
Prediction	foresight/predictor.py	Produces tier, effort, blast radius, rationale, and risk factors from the task description.	Establishes an expected cost and scope baseline before any agent action exists.
Session control	foresight/session_monitor.py	Watches file count, repeated diffs, and cost ceilings; emits signals and reassesses drift.	Session-level behavior cannot be decided from one action alone.
Policy	sentry/policy/engine.py	Converts scope to Policy, makes local decisions first, and uses structured model review only for ambiguity.	Keeps common security decisions deterministic, cheap, and auditable.
Enforcement	sentry/proxy/interceptor.py	Temporarily intercepts file, subprocess, delete, and requests calls in the current Python process.	Places the decision immediately before the original operation can execute.
Telemetry	sentry/eventbus/server.py	Broadcasts actions, costs, predictions, and monitor signals over WebSocket; exports JSON audit logs.	Decouples producers from the dashboard and leaves a machine-readable record.
Presentation	sentry/web/	Renders live connection state, action feed, policy context, cost, and kill state.	Lets a reviewer understand a session without reading Python logs.

The common event contract

Every intercepted operation becomes the same Action record:

Action
  type: file_read | file_write | file_delete | subprocess | network
  target: path, command, or host:port
  payload: mode, unified diff, cwd/env summary, or request-size summary
  timestamp
  agent_session_id

Policy evaluation returns a Verdict with severity, reason, and allowed. This normalized pair is what the proxy enforces, the monitor observes, the event bus broadcasts, and the dashboard displays. Model use is represented separately as CostEvent(model, input_tokens, output_tokens, cost_usd, timestamp).

Four failure modes, four distinct mechanisms

Foresight deliberately gives each failure mode a distinct trigger, reason, event name, and escalation path instead of treating all anomalous activity as one generic warning:

1. Forbidden-pattern interception is deterministic and zero-latency. SentryProxy blocks an action such as .env access before it executes, emits the policy Verdict, and terminates only on a critical denied verdict. This is security enforcement, not prediction.
2. Scope explosion is predictive and contextual. SessionMonitor compares touched files with the predicted blast radius, emits scope_explosion, and re-runs complexity assessment with session context before committing to termination. A wide predicted task is not falsely killed for having a wide footprint.
3. Hard action ceiling is a deterministic workflow cap. The predictor maps isolated to 15 actions, moderate to 40, and wide to 100. Reaching the cap emits hard_ceiling_reached with hard_action_ceiling_reached as its final reason and terminates immediately, even if the agent believes it is nearly done.
4. Parallel fan-out detects a different failure shape from repeated loops: five distinct file or subprocess actions inside three seconds. It emits parallel_explosion and triggers soft reassessment, allowing deliberate, paced parallel work to remain unblocked.

Burn-rate forecasting is separately broadcast as burn_forecast, but it is intentionally informational rather than a kill switch. After enough cost events, it projects minutes remaining at the current cost/minute rate so a user can intervene before a budget ceiling is reached. This keeps forecast visibility separate from hard enforcement.

The independent paralysis_loop check remains in SentryProxy: a rolling 60-second window blocks the sixth identical action after five prior repeats before any model evaluation. It is distinct from parallel fan-out because one catches repeated identical work while the other catches a burst of different work.

This distinction is why the demo can truthfully show two independent outcomes: an immediate .env block from deterministic policy enforcement, and an escalated_for_review result after predictive scope drift is reassessed.

Cost-aware model use

scope_to_policy() and predict_complexity() use structured output. evaluate_action() uses GPT-5.6-terra only when a local rule cannot resolve an in-scope but suspicious action. Each response has an explicit output cap, a stable prompt prefix for caching, and usage recorded through CostTracker.

Predicted effort is calibrated separately from tier: mechanical isolated work routes to low, normal scoped coding work defaults to medium, and high is reserved for genuinely high-stakes or architectural work. A high-effort prediction with a non-wide blast radius carries a deterministic advisory against casually escalating to max or ultra; the CLI renders it as a dim note and the dashboard renders it beside the predicted effort. No extra model call is required for that advice.

The architecture therefore treats model judgement as a metered dependency rather than an invisible background call.

Current prototype boundaries

The repository is intentionally a focused vertical slice, not a claim of production multi-tenant infrastructure:

• Interception is process-scoped Python monkey-patching, not an operating-system sandbox.
• Audit events are held in memory and persisted through explicit JSON export rather than a database.
• Budget ceilings are current heuristics, not learned per-user limits.
• A deployed dashboard needs a reachable event-bus endpoint; it cannot receive events from a developer's local process without a connection path.
• The dashboard is intentionally a single deploy target at sentry/web/; a production hardening pass would add deployment-specific event-bus connectivity and persistence.

What we got wrong, and what it taught us

We originally wrote the demo around the task “Refactor the login flow to add MFA support,” assuming it would produce a narrow terra / isolated prediction so the scripted scope-explosion sequence could exercise escalation. When we ran it against the real GPT-5.6-terra model, it classified MFA and authentication work as security-sensitive and returned sol / high / wide. That was the right call: MFA changes can affect identity, recovery, and session boundaries. Our test assumption, not the prediction, was wrong. The problem was that a wide blast radius has no file-count ceiling to exceed, so the demo narrative no longer matched the model’s real risk assessment. We redesigned the task as a genuinely narrow isolated variable rename, which keeps the expected route and scripted drift consistent. We also added a second, prediction-independent catch: the forbidden-pattern block on .env access. It works regardless of the predicted tier, so the demo now proves both predictive scope monitoring and deterministic interception without guessing a model outcome. This is the same principle the system is built to detect in agents: the gap between what we assumed a task’s scope was and what it actually turned out to be.

Why not just use a bigger model for everything?

Sol costs five times Luna’s input and output price, while many coding tasks are mechanical enough to complete safely with a smaller tier. Defaulting to the frontier tier therefore creates exactly the avoidable spend Foresight is designed to expose. Prediction is not free: it adds a metered call before work begins. But a conservative prediction only adds an escalation round-trip, because Foresight reassesses live drift instead of trusting the first estimate blindly.

Quickstart

Install the Python runtime and CLI:

pip install -r requirements.txt
# or, for the `foresight` command:
pip install -e .

Create your local environment file from the committed template:

copy .env.example .env

Set OPENAI_API_KEY in .env, then run the end-to-end demo:

python -m foresight.demo_scenario

For a complete local replay with zero OpenAI API calls (using real captured GPT-5.6 policy and prediction responses), run:

python -m foresight.demo_scenario --replay

The live path above remains the primary demonstration. Replay is the accessible fallback for judges who want to exercise the real interception and monitoring flow without an API key. For a zero-setup visual tour of the dashboard UI, visit the deployed dashboard; it is distinct from replay mode, which runs the local Python enforcement logic.

CLI workflow:

foresight init
foresight run -- python your_agent_task.py
foresight status
foresight export

Run the dashboard locally:

cd sentry/web
npm install
npm run dev

Open http://localhost:3000/dashboard. The event bus listens on ws://localhost:8765 by default.

Cost tracking and prompt caching

Every OpenAI response becomes a CostEvent, calculated from input/output token usage and streamed to the event bus. Calls use a stable system/context prefix with only task-specific content appended, improving prompt-cache reuse. Clear forbidden or out-of-scope actions are decided locally first, avoiding unnecessary model calls.

Every prediction and policy call is cost-tracked in real time and streamed to the dashboard—Foresight audits its own spend, not just the agent’s.

Codex vs Antigravity build split

• Codex: reasoning-critical implementation in sentry/proxy/interceptor.py, sentry/policy/engine.py, foresight/predictor.py, foresight/session_monitor.py, plus the event bus, CLI, and end-to-end demo.
• Antigravity: frontend polish in sentry/web/, dashboard bug fixes, deployment configuration, environment/documentation cleanup, and this submission README.

This split is intentional and documented per hackathon submission requirements—Codex built 100% of the reasoning-critical logic: prediction, policy evaluation, and escalation.

Roadmap

• Multi-agent swarm monitoring with a shared session risk view.
• Historical session learning to improve tier prediction over time.
• Monitoring support for Claude Code and other agent runtimes beyond Codex.
• Team-level policy profiles and budget guardrails.

License and attribution

MIT License. Built for Build Week, July 2026.