Overrule

Don't ship AI you can't govern.

Runtime policy enforcement for LLM applications — intercept every call, enforce policies, block violations, and ship structured audit events to your cloud dashboard. One SDK. Sub-millisecond. EU AI Act ready.

Quickstart • Features • Architecture • API • Performance • Development

The Problem

Teams shipping AI to production face:

• No runtime guardrails — LLM calls go live unchecked, PII leaks to model providers
• Invisible AI decisions — no audit trail of what the model said, what policies applied, or what was blocked
• Injection vulnerabilities — prompt injection and SQL injection attacks reach production without detection
• Compliance theater — PDF policies and Notion docs that don't actually enforce anything at runtime
• EU AI Act enforcement — Articles 13/14/15 require runtime logging, human oversight, and accuracy monitoring starting August 2026. Fines up to €35M / 7% revenue.

Existing solutions are either enterprise GRC platforms ($50k+/yr), manual review processes, or non-existent for actual runtime enforcement.

The Solution

Overrule is a Python SDK that wraps any LLM call with policy enforcement, violation detection, and structured audit events — all in under 1 millisecond.

from overrule import Guard

async with Guard() as guard:
    response = await guard.chat(
        model="gpt-4o",
        messages=[{"role": "user", "content": user_input}],
        policies=["pii-detection", "injection-detection", "toxicity-detection"],
    )

That's it. Every call is now scanned for PII, injection attacks, and toxic content. Violations are blocked before reaching users, and a structured event is shipped to your cloud dashboard.

Features

For AI Engineers

For Platform Teams

For Compliance

Quickstart

Installation

pip install overrule               # Core SDK
pip install overrule[openai]       # + OpenAI provider
pip install overrule[anthropic]    # + Anthropic provider
pip install overrule[all]          # All providers

Configuration

export OVERRULE_API_KEY=sk_ovr_your_key_here   # from https://overrule.dev/dashboard
export OPENAI_API_KEY=sk-...                    # your LLM provider key

That's all you need. The SDK auto-connects to https://overrule.dev/api and streams events to your dashboard.

Or configure programmatically:

from overrule import Guard, GuardConfig

guard = Guard(config=GuardConfig.from_env(api_key="sk_ovr_xxxxx", fail_open=True))

Basic Usage

from overrule import Guard

async with Guard() as guard:
    response = await guard.chat(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello, what's the weather?"}],
        policies=["pii-detection", "injection-detection"],
    )
    # ✓ Policies evaluated locally (sub-ms typical)
    # ✓ Injection attempts blocked (raises ViolationError)
    # ✓ Other violations surfaced in response.violations
    # ✓ Event streamed to dashboard

    if response.flagged:
        print(f"Violations: {response.violations}")

Verify Your Integration

Run this after installing to confirm events reach your dashboard:

python -c "
import asyncio
from overrule import Guard

async def verify():
    async with Guard() as guard:
        result = await guard.evaluate('test@email.com SSN 123-45-6789', policies=['pii-detection'])
        print(f'PII detected: {len(result.violations)} violations')
        await guard._reporter._flush()
        print('✓ Events sent — check https://overrule.dev/dashboard')

asyncio.run(verify())
"

Environment Variables

How It Works

┌─────────────────────────────────────────────────────────────┐
│                      Your Application                        │
│                                                              │
│  response = await guard.chat(model=..., policies=[...])     │
└──────────────────────────────┬──────────────────────────────┘
                               │
                    ┌──────────▼──────────┐
                    │    Overrule Guard    │
                    │                     │
                    │  1. Input policies  │
                    │  2. LLM call        │
                    │  3. Output policies │
                    │  4. Event ship      │
                    └──────────┬──────────┘
                               │
          ┌────────────────────┼────────────────────┐
          │                    │                    │
┌─────────▼──────┐  ┌─────────▼──────┐  ┌─────────▼──────┐
│  Policy Engine │  │   LLM Provider │  │  Event Buffer  │
│  (local, <1ms) │  │   (OpenAI /    │  │  (async ship   │
│                │  │    Anthropic)  │  │   to cloud)    │
│  PII Detection │  │                │  │                │
│  Injection Det │  │                │  │  10K bounded   │
│  Toxicity Det  │  │                │  │  Backoff retry │
│  Custom Rules  │  │                │  │                │
└────────────────┘  └────────────────┘  └───────┬────────┘
                                                │
                                     ┌──────────▼──────────┐
                                     │  Overrule Cloud     │
                                     │  POST /api/v1/events│
                                     │                     │
                                     │  Dashboard, Alerts, │
                                     │  Compliance Reports │
                                     └─────────────────────┘

Key design decisions:

API Reference

Guard

from overrule import Guard, SyncGuard

# Async (recommended)
async with Guard() as guard:
    response = await guard.chat(model, messages, policies)

# Sync
with SyncGuard() as guard:
    response = guard.chat(model, messages, policies)

guard.chat()

Intercept an LLM call with policy enforcement.

response = await guard.chat(
    model="gpt-4o",
    messages=[{"role": "user", "content": "..."}],
    policies=["pii-detection", "injection-detection"],
    provider="openai",  # or "anthropic"
)

guard.stream()

Streaming interception with token-by-token policy evaluation.

async with Guard() as guard:
    stream = await guard.stream(
        model="gpt-4o",
        messages=[{"role": "user", "content": "..."}],
        policies=["pii-detection", "toxicity-detection"],
        eval_interval=10,  # evaluate every N chunks
    )
    async for chunk in stream:
        print(chunk, end="", flush=True)
    # Violations detected incrementally and at completion
    # ViolationError raised if action=BLOCK

guard.evaluate()

Standalone content evaluation without making an LLM call.

result = await guard.evaluate(
    "My SSN is 123-45-6789",
    policies=["pii-detection"]
)

result.passed       # False
result.violations   # [Violation(policy_id="pii-detection", pattern="ssn", ...)]

@guard.protect()

Decorator for function-level enforcement.

from overrule import Guard, PolicyAction

guard = Guard()

@guard.protect(policies=["injection-detection"], action=PolicyAction.BLOCK)
async def query_database(sql: str) -> str:
    return await db.execute(sql)

guard.register_policy()

Register custom policies.

from overrule.policies.base import BasePolicy, PolicyResult
from overrule.models.violation import Violation

class TopicRestriction(BasePolicy):
    policy_id = "topic-restriction"

    def evaluate(self, content: str, *, direction: str = "input") -> PolicyResult:
        if "medical advice" in content.lower():
            return PolicyResult(
                passed=False,
                violations=[Violation(
                    policy_id=self.policy_id,
                    severity="high",
                    description="Medical advice is restricted",
                )],
            )
        return PolicyResult(passed=True, violations=[])

guard.register_policy(TopicRestriction)

Built-in Policies

Policy Actions

Important: Prompt injection violations always block regardless of default_action (they set violation.blocked=True). To block on all policy violations, set default_action=PolicyAction.BLOCK.

Integrations

LangChain

from overrule.integrations import OverruleCallback

callback = OverruleCallback(
    policies=["pii-detection", "injection-detection", "toxicity-detection"],
    action=PolicyAction.BLOCK,
    on_violation=lambda v: alert_team(v),  # optional hook
)

# Drop into any LangChain LLM
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o", callbacks=[callback])
result = llm.invoke("Hello world")  # automatically governed

Performance

Security Model

Enforcement Behavior

Out of the box, Overrule operates in WARN mode: violations are detected, logged to telemetry, and surfaced in response.violations — but the LLM call proceeds. This lets you integrate safely without breaking existing flows.

Exception: Prompt injection violations always block, regardless of default_action. They set violation.blocked=True, which triggers ViolationError before the LLM is called.

To enforce hard blocking on all violations:

from overrule import Guard, PolicyAction

guard = Guard(default_action=PolicyAction.BLOCK)

Fail-Open Architecture

By default, fail_open=True: if the SDK crashes during policy evaluation, the LLM call proceeds unguarded rather than failing your application. This is a deliberate availability-over-security tradeoff.

For strict enforcement where governance failure = application failure:

guard = Guard(fail_open=False)

Streaming Limitations

guard.stream() evaluates policies incrementally (every N chunks). Tokens yielded before a violation is detected cannot be recalled. For strict enforcement where no violating content may reach the user, use non-streaming guard.chat() with default_action=BLOCK.

Content Scanning

Large inputs (>100KB) are sampled: head, middle, and tail windows are scanned. A WARNING log is emitted when truncation occurs. The sampling strategy covers all three regions to prevent evasion by placing payloads in the center.

Other Protections

• API keys never exposed in repr(), str(), or serialized output
• PII redaction shows only last 4 characters (no BIN/prefix leakage)
• Config values are bounds-validated (batch_size, flush_interval, etc.)
• PEP 561 compliant (py.typed marker for downstream type checking)
• No secrets in logs — all sensitive values masked in debug output
• Policy timeout (5s) prevents ReDoS from triggering fail-open bypass

Cloud Dashboard

The Overrule cloud dashboard at overrule.dev provides:

Plans

Project Structure

overrule-sdk/
├── overrule/
│   ├── __init__.py              # Public API (Guard, SyncGuard, PolicyAction, policies)
│   ├── guard.py                 # Core Guard class (async context manager, REDACT flow, streaming)
│   ├── stream.py                # StreamGuard — token-by-token policy eval for streaming
│   ├── sync.py                  # SyncGuard wrapper (background thread + event loop)
│   ├── exceptions.py            # Exception hierarchy (ViolationError, TransportError, etc.)
│   ├── logging.py               # Structured logging utilities
│   ├── integrations/
│   │   ├── __init__.py          # Framework integration exports
│   │   └── langchain.py         # OverruleCallback for LangChain
│   ├── models/
│   │   ├── config.py            # GuardConfig + PolicyAction enum (BLOCK, LOG, WARN, REDACT)
│   │   ├── event.py             # InterceptEvent (structured governance event)
│   │   └── violation.py         # Violation model (policy_id, severity, direction)
│   ├── policies/
│   │   ├── base.py              # BasePolicy abstract class + PolicyResult
│   │   ├── registry.py          # Thread-safe PolicyRegistry
│   │   ├── pii.py               # PII detection (7 patterns, raw_match metadata)
│   │   ├── injection.py         # Prompt injection (8) + SQL injection (5) patterns
│   │   └── toxicity.py          # Toxicity detection (profanity, slurs, violence, 3 tiers)
│   └── transport/
│       ├── reporter.py          # Async EventReporter (batching, backoff, circuit breaker)
│       └── dead_letter.py       # Dead-letter queue (persist dropped events to disk)
├── tests/                       # 137 tests (pytest)
├── examples/                    # Runnable integration examples
├── pyproject.toml               # Build config + dependencies
├── CHANGELOG.md                 # Version history
└── LICENSE                      # MIT

Compliance Mapping

Roadmap

• Core Guard with fail-open architecture
• PII detection policy (credit cards, SSN, email, phone, IBAN, passport, IPv4)
• Injection detection policy (8 prompt injection + 5 SQL injection patterns)
• Async + Sync APIs (Guard + SyncGuard)
• Multi-provider support (OpenAI + Anthropic)
• Custom policy engine (BasePolicy interface)
• Decorator API (@guard.protect())
• Standalone evaluation (guard.evaluate())
• Circuit breaker (5 failures → open → 30s cooldown → recovery)
• Bounded event buffer (10K max, graceful shutdown flush)
• Exponential backoff with jitter on transport failures
• Cloud event streaming (POST /api/v1/events)
• Environment-based configuration
• Published on PyPI (pip install overrule)
• Toxicity detection policy (profanity, slurs, violence, 3 severity tiers)
• REDACT action (replace violations with tokens instead of blocking)
• Output policy enforcement (response scanning)
• Streaming interception (guard.stream() with incremental evaluation)
• LangChain integration (OverruleCallback drop-in handler)
• Dead-letter queue (failed events persisted to disk, auto-recovered)
• Policy hot-reload (update policies at runtime without restart)
• Credit card detection with dash/space formats
• 137-test suite (pytest)
• PEP 561 compliant (py.typed)
• CrewAI integration (agent-level governance)
• OpenAI Agents SDK wrapper
• Rust core for <100μs evaluation
• Policy marketplace (community-contributed policies)

Examples

The examples/ directory contains runnable scripts for common use cases:

# Run any example
cd overrule-sdk
export OVERRULE_API_KEY=sk_ovr_...
python examples/evaluate_only.py

Development

# Clone
git clone https://github.com/overruledev/overrule-sdk.git
cd overrule-sdk

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest --cov=overrule --cov-fail-under=80

# Lint + format check
ruff check .
ruff format --check .

# Type check (strict)
mypy overrule/

CI/CD Pipeline

Every push and PR triggers a production-grade CI pipeline:

PR Quality Gates (run on pull requests only):

• New dependency detection with review notice
• Debug print() statement detection
• TODO/FIXME/HACK tracker
• Secret pattern scanning (hard fail)
• .env file leak detection (hard fail)
• Version bump notification

Contributing

We're building in public. Contributions welcome.

# Fork + clone
git clone https://github.com/yourusername/overrule-sdk.git

# Create feature branch
git checkout -b feature/your-feature

# Make changes, then run the full CI suite locally
pytest --cov=overrule --cov-fail-under=80   # Tests + coverage
ruff check .                                 # Lint
ruff format --check .                        # Format
mypy overrule/                               # Type check

git commit -m "feat: your feature description"
git push origin feature/your-feature

All PRs must pass lint, typecheck, security audit, tests (80%+ coverage), and build verification before merge.

Contact

License

MIT License. See LICENSE for details.

Built for teams shipping AI to production.
_{Overrule — because governance shouldn't slow you down.}