ShadowAudit

Deprecated: ShadowAudit has been renamed to CapFence. New projects should install capfence and use the maintained repository: https://github.com/capfencelabs/capfence

Deterministic runtime authorization for AI agent tool calls.

ShadowAudit sits between AI agents and their tools. It evaluates every tool call against deterministic policy before execution, then allows it, blocks it, or requires approval.

It is closer to IAM, Open Policy Agent, API gateways, and admission controllers than prompt guardrails or moderation.

Agent -> ShadowAudit -> Tool
          |
          +-- allow
          +-- deny
          +-- require approval

Why This Exists

Agents increasingly call tools that can move money, edit databases, run shell commands, read files, modify permissions, and operate SaaS admin APIs.

Prompt instructions are not an execution boundary. ShadowAudit gives those tool calls an explicit runtime authorization layer:

• No LLM call in the gate path.
• Policy-as-code decisions.
• Default-deny behavior when policy does not match.
• Fail-closed handling for policy and audit failures.
• Local audit logs with hash-chain verification.
• Observe mode for safe rollout before enforcement.

Install

pip install capfence

This shadowaudit package remains available only as a deprecated compatibility release for existing users. It will emit a warning at import time.

60-Second Example

Create a policy:

deny:
  - capability: shell.execute
    contains: "rm -rf"

require_approval:
  - capability: payments.transfer
    amount_gt: 1000

allow:
  - capability: shell.execute
  - capability: payments.transfer
    amount_lte: 1000

Evaluate a tool call before execution:

from shadowaudit.core.gate import Gate

gate = Gate()

result = gate.evaluate(
    agent_id="ops-agent",
    task_context="shell",
    risk_category="shell_execution",
    capability="shell.execute",
    policy_path="policies/shell_agent.yaml",
    payload={"command": "rm -rf /var/lib/postgresql"},
)

if not result.passed:
    raise PermissionError(f"Blocked: {result.reason}")

The dangerous command never reaches the tool.

Framework Integrations

ShadowAudit can wrap tools in:

• LangChain
• LangGraph
• CrewAI
• OpenAI Agents SDK
• MCP
• PydanticAI
• LlamaIndex
• AutoGen
• Direct Python runtimes

LangChain example:

from shadowaudit import ShadowAuditTool
from langchain.tools import ShellTool

safe_shell = ShadowAuditTool(
    tool=ShellTool(),
    agent_id="ops-agent",
    capability="shell.execute",
    policy_path="policies/shell_agent.yaml",
)

CLI Workflows

Scan for ungated tools:

shadowaudit check ./src --fail-on-ungated

Validate a policy:

shadowaudit check-policy policies/shell_agent.yaml

Replay a trace through policy:

shadowaudit simulate --trace-file traces/agent_trace.jsonl --compare

Verify audit-log integrity:

shadowaudit verify --audit-log audit.db

Rollout Path

1. Start in observe mode and log decisions without blocking.
2. Review audit logs and tune policies.
3. Enforce policy for high-risk tools.
4. Add CI checks so new ungated tools cannot quietly ship.
5. Replay incidents and policy changes against saved traces.

What ShadowAudit Is Not

ShadowAudit is a runtime authorization and audit layer. It does not replace:

• sandboxing for shell/code execution
• least-privilege credentials
• network egress controls
• prompt-injection defenses
• human review for genuinely ambiguous high-risk actions

Use it as the deterministic control point before tool execution.

Why Not Prompt Guardrails?

Prompt guardrails are useful, but they do not enforce execution. A prompt can be bypassed, misinterpreted, or ignored under pressure. ShadowAudit adds a deterministic enforcement boundary that blocks tool calls before they execute and records a tamper-evident audit trail.

Where It Sits In Your Stack

Agent framework -> ShadowAudit gate -> Tool/API/DB/Shell

ShadowAudit does not replace sandboxing, network egress controls, or least-privilege credentials. It complements them by enforcing runtime policy at the tool boundary.

Project Status

ShadowAudit is beta infrastructure for agent tool governance. The repo includes:

• deterministic gate and policy engine
• local audit log with hash-chain verification
• approval workflows
• observe mode and bypass audit trails
• framework adapters
• MCP gateway and adapter
• static scanner and CI mode
• OWASP Agentic Top 10 and EU AI Act evidence reports
• typed Python package with ruff, mypy, and pytest coverage

Current local verification: run pytest -q.

Documentation

• Docs: https://capfence.dev/
• CapFence PyPI: https://pypi.org/project/capfence/
• Deprecated ShadowAudit PyPI: https://pypi.org/project/shadowaudit/
• Repository: https://github.com/capfencelabs/capfence

Useful starting points:

• Quickstart
• First policy
• Recipes
• Compatibility matrix
• Protect shell tools
• Protect payment agents
• Secure MCP servers
• Demo walkthrough
• Demo cast
• Policy schema

Contributing

git clone https://github.com/capfencelabs/capfence.git
cd capfence
pip install -e ".[dev]"
pytest tests/ -q

Policy recipes, framework adapters, taxonomies, docs, and focused bug reports are welcome.

License

MIT License

_{Built by CapFence Labs}