👁️ PANOPTIS

(formerly AgentWatch)

The All-Seeing Pre-Execution Guardrail & Invariant Engine for Autonomous AI.

AI agents are prone to silent failures, ungrounded loops, and unintended side effects.
Panopti acts as an active, pre-execution guardrail — auditing reasoning and blocking risky actions before they hit your production stack.

pip install panoptis

# 🎮 Instant Interactive Terminal Demo (Simulates rogue agent attempting DROP TABLE users;)
panopti demo

# 🌐 Launch Zero-Dependency Local Web Playground
panopti playground

Decide on every tool call before it runs, and keep verifiable evidence of what happened.

Open in Colab · Interactive Demo · Quick Start · How It Works · Architecture · Discord · Contribute

What blocks and what only observes. The v3 evidence engine (panoptis observe / inspect / compare / replay) is a passive instrument: it reconstructs, compares, replays and explains AI-native systems from immutable evidence, and it never blocks anything. Enforcement comes from the protection pipeline (panoptis.protect): it decides each tool call before it runs (allow, block, ask a human, or rewrite), sanitizes results before the agent sees them, and records tamper-evident (Merkle-sealed) evidence. It is reached through protect_tool(), watch(agent, protection=...), LangChain/LangGraph/AutoGen/smolagents tools, the Claude Code PreToolUse hook and the MCP gateway; some other adapters only observe. The exact boundary of every integration is in docs/ADAPTER_SUPPORT.md, the security model in docs/THREAT_MODEL.md, and every claim's evidence in docs/CLAIMS.md.

panoptis observe python examples/research_system.py   # record a program
panoptis inspect latest                                 # reconstructed execution
panoptis provenance report.md --run latest              # evidence lineage
panoptis compare latest~1 latest                        # earliest divergence
panoptis replay latest --level L2                       # replay with captured results

See docs/v3/USER_GUIDE.md, docs/v3 (architecture, ADRs, migration) and AWBench results. Most analysis features are EXPERIMENTAL; panoptis status shows each capability's maturity. The v0.2 guardrail features described below remain available during migration.

The Problem Nobody Is Solving

Agent runs.
Output looks correct.
Database is corrupted.
You find out 3 hours later when a customer complains.

1 in 20 AI agent requests fail silently in production.

Current observability tools — Langfuse, Phoenix, Datadog — tell you what happened after it happened. By then the damage is done.

The real problem: an agent that confidently fails is indistinguishable from an agent that correctly succeeds — unless you have a layer watching the reasoning, not just the output.

That layer didn't exist. Until now.

💡 How It Works

AgentWatch sits between your agent and the world, intercepting every action before it executes.

flowchart LR
    A["🤖 Your Agent"] --> AW

    subgraph AW["🛡️ AgentWatch"]
        direction TB
        S1["1 . Capture<br/>every reasoning step"]
        S2["2 . Score<br/>independent model audit"]
        S3["3 . Block<br/>dangerous actions pre-execution"]
        S1 --> S2 --> S3
    end

    AW -->|"✅ allowed"| W["🌍 The World<br/>tools · APIs · databases"]
    AW -.->|"⛔ blocked"| X["🚫 Action vetoed<br/>+ alert fired"]

    style A fill:#1e293b,stroke:#3ecf8e,color:#fff
    style AW fill:#0f172a,stroke:#3ecf8e,color:#fff
    style W fill:#1e293b,stroke:#3ecf8e,color:#fff
    style X fill:#3b1212,stroke:#ef4444,color:#fff

The key insight: an agent scoring its own reasoning is structurally biased toward overconfidence. It almost always thinks it did well — even when it didn't.

AgentWatch deploys a second model, architecturally separate, with no access to the agent's own reasoning trace. Its only job: find failure before the next action fires.

📸 Screenshots

Pre-execution blocking, not post-hoc logging. A blocked rm -rf /tmp/* with live confidence (0.18) and blast-radius (HIGH) read out before it ever runs.	Your Agent → AgentWatch → The World. Capture every reasoning step, score it with an independent model, block dangerous actions before they run.
Six production-grade modules. Reasoning Auditor · Safety Engine · One-Click Rollback · Multi-Agent DAG · Causal Memory · Compliance Ready.	What nobody else ships. Pre-execution blocking, independent audit, git-backed rollback — and it wraps any agent framework in 60 seconds.

🏗️ Architecture

AgentWatch is built as a layered system — adapters feed a normalized event stream into the core engine, which fans out to safety, reasoning, rollback, and observability subsystems.

flowchart TB
    subgraph ADAPTERS["🔌 Framework Adapters"]
        direction LR
        LC["LangChain"]
        CR["CrewAI"]
        AG["AutoGPT"]
        CC["Claude Code"]
        LG["LangGraph"]
        AU["AutoGen"]
    end

    ADAPTERS --> BUS["📡 Event Bus<br/>normalized AgentEvent stream"]

    BUS --> CORE

    subgraph CORE["⚙️ Core Engine"]
        direction TB
        SAFE["🛡️ Safety Engine<br/>15 legacy patterns · blast radius"]
        REASON["🧠 Reasoning Auditor<br/>independent scoring"]
        SAFE --- REASON
    end

    CORE --> ROLL["⏪ Rollback<br/>git-backed checkpoints"]
    CORE --> MEM["💾 Causal Memory<br/>cross-session trails"]
    CORE --> COST["💰 Cost Governance<br/>per-session budgets"]
    CORE --> GOV["📋 Compliance<br/>GDPR · HIPAA · EU AI Act"]

    CORE --> TEL["🔭 Telemetry<br/>OpenTelemetry spans"]
    TEL --> API["🌐 REST + WebSocket API"]
    API --> DASH["📊 Live Dashboard<br/>Next.js"]

    style ADAPTERS fill:#0f172a,stroke:#64748b,color:#fff
    style CORE fill:#0f172a,stroke:#3ecf8e,color:#fff
    style BUS fill:#1e293b,stroke:#3ecf8e,color:#fff
    style API fill:#1e293b,stroke:#3ecf8e,color:#fff
    style DASH fill:#1e293b,stroke:#3ecf8e,color:#fff

CLI Architecture

• All shell commands executed by the CLI are routed through a secure, validated wrapper located in panoptis/cli/_utils to prevent injection vulnerabilities.

🔐 Security Pipeline

Every TOOL_CALL event runs the full gauntlet before it is ever allowed to touch the outside world. Blocking happens pre-execution — not as a post-hoc log entry.

flowchart TD
    EV["📥 TOOL_CALL event"] --> CHK{"event is<br/>a tool call?"}
    CHK -->|no| PASS["↩️ pass through unchanged"]
    CHK -->|yes| AUDIT["🧠 Reasoning audit<br/>confidence score 0.0 – 1.0"]

    AUDIT --> MATCH["🔍 Pattern match<br/>15 legacy risk patterns"]
    MATCH --> NORM["🧹 Intent normalizers<br/>rm · disk · perms · RCE pipe"]
    NORM --> BLAST["💥 Blast-radius estimate"]

    BLAST --> DECIDE{"policy<br/>decision"}
    DECIDE -->|block| BLOCK["⛔ BLOCKED<br/>status set · alert fired"]
    DECIDE -->|approval| HITL{"human<br/>approves?"}
    DECIDE -->|allow| ALLOW["✅ ALLOWED<br/>action proceeds"]

    HITL -->|yes| ALLOW
    HITL -->|no / no callback| BLOCK

    style EV fill:#1e293b,stroke:#3ecf8e,color:#fff
    style BLOCK fill:#3b1212,stroke:#ef4444,color:#fff
    style ALLOW fill:#0f2a18,stroke:#3ecf8e,color:#fff
    style HITL fill:#2a230f,stroke:#eab308,color:#fff

Blocked by default: rm -rf / · curl | bash · disk formatting · credential exfiltration · DROP TABLE · mass deletion · privilege escalation. The legacy SafetyEngine gate ships 15 built-in patterns (5 critical, blocked by default); the protection pipeline (panoptis.protect) adds about 70 finding types across shell, SQL, filesystem, network/SSRF, Python code, secrets and prompt injection. Measured on a synthetic corpus in benchmarks/results/protect_eval/REPORT.md (with its limits stated there).

The intent normalizers are what make this robust — they catch bypass attempts the naive regex misses: split flags (rm -r -f /), long-form flags (rm --recursive --force /), and reordered fetch-then-interpret RCE chains.

🔄 Reasoning Audit Sequence

On the legacy async watch() path, the auditor scores each step before the action runs; a drop below threshold holds the next action and fires an alert. The scoring model needs ANTHROPIC_API_KEY; without it the audit is skipped and only pattern rules apply.

sequenceDiagram
    participant Agent
    participant AgentWatch
    participant Auditor as Independent Auditor
    participant World as Tools / APIs / DB

    Agent->>AgentWatch: emit reasoning step + tool call
    AgentWatch->>Auditor: score_step(step)
    Auditor-->>AgentWatch: confidence · hallucination risk · goal drift

    alt confidence below threshold
        AgentWatch--xAgent: HOLD action + fire alert
        Note over AgentWatch: human decides next move
    else confidence ok
        AgentWatch->>World: action proceeds
        World-->>Agent: result
        AgentWatch->>AgentWatch: checkpoint for rollback
    end

🚀 Quick Start

Run the interactive Panopti Pre-Execution Guardrails Tutorial directly in Google Colab with zero local installation, or follow the local setup below:

# Install
pip install panoptis

# Configure environment variables
# Copy the template and set AGENTWATCH_API_KEY (docker compose requires it), e.g.
#   python -c "import secrets; print(secrets.token_urlsafe(32))"
cp .env.example .env

# Start the dashboard
docker compose up -d

# Wrap your agent
panoptis session watch "Build me a REST API"

Dashboard → http://localhost:3000 API Docs → http://localhost:8000/docs

That's it. Set AGENTWATCH_API_KEY in .env (compose refuses to start without it); everything else has working defaults, customizable via .env.example. The v3 evidence store lives on the Postgres service, so it survives container restarts.

🔌 Supported Frameworks

AgentWatch wraps your existing agent. You change nothing. Detailed guides for each framework live in the docs/adapters/ directory.

✨ Core Features

🧠 Reasoning Auditor

The feature nobody else has built.

from panoptis.reasoning.auditor import ReasoningAuditor

auditor = ReasoningAuditor()
audit = await auditor.audit_step(step.step_number, step)

print(audit.score)        # 0.0 – 1.0 confidence in the step
print(audit.rationale)    # why the auditor scored it this way

When the score drops below your threshold, the next action is held — not logged after the fact. An alert fires. You decide what happens next.

🛡️ Safety Engine

from panoptis.core.safety import SafetyEngine
from panoptis.core.schema import ExecutionStatus

engine = SafetyEngine()
checked = await engine.check_event(event)

if checked.status == ExecutionStatus.BLOCKED:
    print(f"Blocked: {checked.safety.reasons}")
    print(f"Risk level: {checked.safety.risk_level.value}")

Blocks dangerous calls pre-execution (15 legacy patterns here; the protection pipeline covers more: docs/THREAT_ANALYSIS.md), not post-hoc logging.

🔒 Secure Subprocess Wrapper

To prevent command-injection risks when invoking external utilities in CLI extensions or scripts, use the central secure command execution utility:

from panoptis.cli._utils import run, CommandError

try:
    # Runs securely (shell=False) with strict argument character whitelisting
    result = run(["echo", "hello world"])
    print(result.stdout)
except CommandError as exc:
    print(f"Command execution failed: {exc}")

⏪ One-Click Rollback

panoptis session rollback <session-id> --to-step 12

Every step is a git-backed filesystem snapshot. Irreversible actions become reversible. Click rollback in the dashboard or use the CLI.

📊 Live Dashboard

Real-time WebSocket stream of every action your agent takes. Confidence meter updating per step. Colour-coded by span type. No polling. No refresh.

💾 Persistent Memory

Cross-session episodic, semantic, and procedural memory backed by a causal graph. Your agent remembers what it decided and why — across restarts, across sessions.

💰 Cost Intelligence

Per-session token budget with hard stop. Real-time spend tracking. Alerts at 80%. Blocks at 100%. Prevents runaway agents from bankrupting you overnight.

🔔 Alerting

Slack + PagerDuty when confidence drops or actions are blocked. Every alert contains full context — not just "something failed."

🌐 REST API

GET  /api/v1/sessions
GET  /api/v1/sessions/{id}/replay
GET  /api/v1/sessions/{id}/confidence
GET  /api/v1/sessions/{id}/checkpoints
POST /api/v1/sessions/{id}/rollback
GET  /api/v1/safety/blocked
GET  /api/v1/dashboard/summary
WS   /ws/events

Full Swagger docs at localhost:8000/docs.

🤝 How AgentWatch Fits Into Your Stack

AgentWatch is designed to run alongside and complement your existing tracing and observability tools rather than replace them.

While platforms like Langfuse, Phoenix, and Datadog are excellent at post-hoc tracing, logging, and performance metrics, AgentWatch focuses specifically on pre-execution guardrails and runtime safety interventions.

Here is how AgentWatch complements your monitoring stack:

🧱 Stack

✅ Verified

• 205/205 tests passing
• docker compose up — zero errors
• API live at localhost:8000
• Dashboard live at localhost:3000
• Claude Code, LangChain, CrewAI, AutoGPT adapters working

🤝 Contributing

AgentWatch is built in the open. Contributors get their name on the landing page after their first merged PR.

flowchart LR
    A["💬 Join Discord<br/>discuss your approach"] --> B["🔍 Pick an issue<br/>good first · intermediate · advanced"]
    B --> C["🔨 Build<br/>fork · branch · code"]
    C --> D["✅ Open PR<br/>auto-tested by CI"]
    D --> E["🎉 Merge<br/>your name on the landing page"]

    style A fill:#0f172a,stroke:#5865F2,color:#fff
    style B fill:#0f172a,stroke:#3ecf8e,color:#fff
    style C fill:#0f172a,stroke:#3ecf8e,color:#fff
    style D fill:#0f172a,stroke:#3ecf8e,color:#fff
    style E fill:#0f2a18,stroke:#3ecf8e,color:#fff

Before you start → join the Discord. Get help picking the right issue, discuss your approach, and ship faster.

Local Backend Setup

git clone https://github.com/Shinraya-Tech-Labs/AgentWatch
cd AgentWatch
docker compose up -d
pip install -e ".[dev]"
pytest tests/

Local Frontend Dashboard Setup

cd frontend
npm install
npm run dev

The dashboard will be live at http://localhost:3000.

Every PR to main is automatically tested by the test-on-pr workflow, which runs the suite with coverage and posts the results as a PR comment.

📦 Release Process

Releases publish to PyPI automatically via the publish-pypi workflow whenever a version tag is pushed:

# Bump [project].version in pyproject.toml to match the tag first, then:
git tag v0.X.Y
git push origin v0.X.Y
# PyPI publishes automatically.

On a v* tag the workflow verifies the tag matches pyproject.toml (failing fast on a mismatch), builds the wheel + sdist, runs twine check, uploads with twine, and creates a GitHub Release titled AgentWatch v0.X.Y — notes pulled from CHANGELOG.md, with the .whl and .tar.gz attached.

One-time setup — add the PyPI token

The upload step authenticates with a PyPI API token stored as a GitHub secret named PYPI_TOKEN:

1. Create a token at pypi.org → Account settings → API tokens (scope it to this project).
2. In the repo: Settings → Secrets and variables → Actions → New repository secret.
3. Name it PYPI_TOKEN and paste your pypi-... token as the value.

🗺️ Roadmap

AgentWatch v0.2.0 is being built now — 90 features across 10 phases including:

• Causal memory graph (cross-session reasoning trails)
• Inter-agent causal DAG (multi-agent failure tracing)
• OWASP Agentic Top 10 scanner
• EU AI Act Article 15 compliance package
• Counterfactual replay ("what if step 3 was different")
• Open reasoning trace schema (the OTEL play)

Every open issue on the roadmap is available to contributors. Browse them here.

👥 Contributors & Open Source Programs

AgentWatch is built in collaboration with a growing community of open-source developers.

We participate actively in student and developer mentorship programs, including:

• SSOC 2026 (Social Summer of Code)
• NSOC 2026 (Nexus Spring of Code)

These programs introduce student developers to production-grade Python projects. As part of their learning journey, students fork the repository, set up local environments, run tests, and submit PRs. This active participation explains the repository's high fork-to-star ratio.

For a full list of program details and the contributors who make AgentWatch possible, please check out our CONTRIBUTING.md.

🔍 Similar & Complementary Projects

We believe in fostering a collaborative, transparent AI engineering ecosystem. If you are building agentic workflows, we recommend checking out these excellent projects to round out your stack:

• Langfuse — Open-source LLM engineering platform for tracing, evaluation, prompt management, and metrics.
• Arize Phoenix — AI observability platform for tracing, evaluating, and troubleshooting LLM applications.
• Arize — Enterprise-grade ML observability for monitoring model performance, drift, and data quality.

💬 Community

Discord — discord.gg/UT9uaeY46e

Contributors discuss issues, get unblocked, and ship together. Get your name on the landing page after your first PR merges!

📄 License

Apache 2.0 — use it, fork it, build on it.