Overrule

Don't ship AI you can't govern.

Runtime policy enforcement for LLM applications — intercept every call, enforce policies, block violations, and ship structured audit events to your cloud dashboard. One SDK. Sub-millisecond. EU AI Act ready.

Quickstart • Features • Architecture • API • Performance • Development

---

The Problem

Teams shipping AI to production face:

• No runtime guardrails — LLM calls go live unchecked, PII leaks to model providers
• Invisible AI decisions — no audit trail of what the model said, what policies applied, or what was blocked
• Injection vulnerabilities — prompt injection and SQL injection attacks reach production without detection
• Compliance theater — PDF policies and Notion docs that don't actually enforce anything at runtime
• EU AI Act enforcement — Articles 13/14/15 require runtime logging, human oversight, and accuracy monitoring starting August 2026. Fines up to €35M / 7% revenue.

Existing solutions are either enterprise GRC platforms ($50k+/yr), manual review processes, or non-existent for actual runtime enforcement.

The Solution

Overrule is a Python SDK that wraps any LLM call with policy enforcement, violation detection, and structured audit events — all in under 1 millisecond.

from overrule import Guard

async with Guard() as guard:
    response = await guard.chat(
        model="gpt-4o",
        messages=[{"role": "user", "content": user_input}],
        policies=["pii-detection", "injection-detection", "toxicity-detection"],
    )

That's it. Every call is now scanned for PII, injection attacks, and toxic content. Violations are blocked before reaching users, and a structured event is shipped to your cloud dashboard.

---

Features

For AI Engineers

Feature	Description
1-Line Integration	Wrap any LLM call with guard.chat(). Works with OpenAI and Anthropic today, more providers coming.
PII Detection	Credit cards, SSN, email, phone, IBAN, passport, IPv4 — intercepted at runtime
Injection Detection	8 prompt injection + 5 SQL injection patterns blocked before they reach the model
Toxicity Detection	Profanity, slurs, hate speech, violence incitement — 3 severity tiers
REDACT Action	Replace violations in output with [POLICY_ID] tokens instead of blocking
Custom Policies	Extend BasePolicy for domain-specific rules (bias, topic restriction, NER)
Multi-Provider	Same governance across OpenAI and Anthropic — swap providers without touching policy logic
Streaming Governance	guard.stream() — token-by-token policy evaluation for streaming LLM calls
LangChain Integration	OverruleCallback — drop-in governance for any LangChain chain or agent
Async + Sync	Guard for async, SyncGuard for synchronous — same API surface
Decorator API	@guard.protect() for function-level enforcement
Standalone Evaluation	guard.evaluate(text) to scan content without making an LLM call
Policy Hot-Reload	guard.reload_policies() — update policies at runtime without restart

For Platform Teams

Feature	Description
Fail-Open Architecture	SDK errors never crash your application. Governance degrades gracefully.
Circuit Breaker	Opens after 5 consecutive failures, 30s cooldown, automatic recovery
Dead-Letter Queue	Failed events persisted to disk, auto-retried on next startup
Bounded Buffer	10K event max buffer with graceful shutdown flush
Exponential Backoff	Jittered retry on transport failures — no thundering herd
Minimal Hot-Path Latency	Policies evaluate locally (<1ms typical prompts, ~30ms on 100KB inputs). Telemetry ships async in background.
Cloud Event Streaming	Governance metadata streamed to Overrule dashboard in real-time (prompts/completions stay local)
Structured Violations	Severity-tagged (info/low/medium/high/critical) with full context and direction
Environment Config	OVERRULE_API_KEY, OVERRULE_ENDPOINT, OVERRULE_FAIL_OPEN — all env-configurable

For Compliance

Feature	Description
EU AI Act Articles 13/14/15	Maps directly to logging, oversight, and accuracy requirements
Structured Audit Trail	Every LLM interaction logged with metadata (model, provider, tokens, latency, policies, violations). Prompts and completions never leave your infrastructure.
Exportable Telemetry	Metadata + violation evidence in structured format for auditors and regulators
Runtime Enforcement	Governance is code, not a document. Prove to regulators what's actually enforced.
Cloud Dashboard	Visual overview at overrule.dev — posture score, events, policies, billing

---

Quickstart

Installation

pip install overrule               # Core SDK
pip install overrule[openai]       # + OpenAI provider
pip install overrule[anthropic]    # + Anthropic provider
pip install overrule[all]          # All providers

Configuration

export OVERRULE_API_KEY=sk_ovr_your_key_here   # from https://overrule.dev/dashboard
export OPENAI_API_KEY=sk-...                    # your LLM provider key

That's all you need. The SDK auto-connects to https://overrule.dev/api and streams events to your dashboard.

Or configure programmatically:

from overrule import Guard, GuardConfig

guard = Guard(config=GuardConfig.from_env(api_key="sk_ovr_xxxxx", fail_open=True))

Basic Usage

from overrule import Guard

async with Guard() as guard:
    response = await guard.chat(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello, what's the weather?"}],
        policies=["pii-detection", "injection-detection"],
    )
    # ✓ Policies evaluated locally (sub-ms typical)
    # ✓ Injection attempts blocked (raises ViolationError)
    # ✓ Other violations surfaced in response.violations
    # ✓ Event streamed to dashboard

    if response.flagged:
        print(f"Violations: {response.violations}")

Verify Your Integration

Run this after installing to confirm events reach your dashboard:

python -c "
import asyncio
from overrule import Guard

async def verify():
    async with Guard() as guard:
        result = await guard.evaluate('test@email.com SSN 123-45-6789', policies=['pii-detection'])
        print(f'PII detected: {len(result.violations)} violations')
        await guard._reporter._flush()
        print('✓ Events sent — check https://overrule.dev/dashboard')

asyncio.run(verify())
"

Environment Variables

Variable	Default	Description
OVERRULE_API_KEY	—	Your API key from overrule.dev dashboard
OVERRULE_ENDPOINT	https://overrule.dev/api	Cloud endpoint for event ingestion
OVERRULE_ENVIRONMENT	production	Environment tag on events
OVERRULE_FAIL_OPEN	true	If true, SDK errors don't crash your app
OVERRULE_BATCH_SIZE	50	Events batched before flush (max 100)
OVERRULE_FLUSH_INTERVAL	5.0	Seconds between background flushes

---

How It Works

┌─────────────────────────────────────────────────────────────┐
│                      Your Application                        │
│                                                              │
│  response = await guard.chat(model=..., policies=[...])     │
└──────────────────────────────┬──────────────────────────────┘
                               │
                    ┌──────────▼──────────┐
                    │    Overrule Guard    │
                    │                     │
                    │  1. Input policies  │
                    │  2. LLM call        │
                    │  3. Output policies │
                    │  4. Event ship      │
                    └──────────┬──────────┘
                               │
          ┌────────────────────┼────────────────────┐
          │                    │                    │
┌─────────▼──────┐  ┌─────────▼──────┐  ┌─────────▼──────┐
│  Policy Engine │  │   LLM Provider │  │  Event Buffer  │
│  (local, <1ms) │  │   (OpenAI /    │  │  (async ship   │
│                │  │    Anthropic)  │  │   to cloud)    │
│  PII Detection │  │                │  │                │
│  Injection Det │  │                │  │  10K bounded   │
│  Toxicity Det  │  │                │  │  Backoff retry │
│  Custom Rules  │  │                │  │                │
└────────────────┘  └────────────────┘  └───────┬────────┘
                                                │
                                     ┌──────────▼──────────┐
                                     │  Overrule Cloud     │
                                     │  POST /api/v1/events│
                                     │                     │
                                     │  Dashboard, Alerts, │
                                     │  Compliance Reports │
                                     └─────────────────────┘

Key design decisions:

Decision	Rationale
Policies evaluate locally	Zero network latency on the hot path
Telemetry ships async	Your app never waits on governance infrastructure
Fail-open by default	A governance SDK that crashes your app is worse than no governance
Circuit breaker	5 failures → open → 30s cooldown → half-open → recover
Bounded buffer	Memory-safe: drops oldest events at 10K rather than OOM

---

API Reference

Guard

from overrule import Guard, SyncGuard

# Async (recommended)
async with Guard() as guard:
    response = await guard.chat(model, messages, policies)

# Sync
with SyncGuard() as guard:
    response = guard.chat(model, messages, policies)

guard.chat()

Intercept an LLM call with policy enforcement.

response = await guard.chat(
    model="gpt-4o",
    messages=[{"role": "user", "content": "..."}],
    policies=["pii-detection", "injection-detection"],
    provider="openai",  # or "anthropic"
)

guard.stream()

Streaming interception with token-by-token policy evaluation.

async with Guard() as guard:
    stream = await guard.stream(
        model="gpt-4o",
        messages=[{"role": "user", "content": "..."}],
        policies=["pii-detection", "toxicity-detection"],
        eval_interval=10,  # evaluate every N chunks
    )
    async for chunk in stream:
        print(chunk, end="", flush=True)
    # Violations detected incrementally and at completion
    # ViolationError raised if action=BLOCK

guard.evaluate()

Standalone content evaluation without making an LLM call.

result = await guard.evaluate(
    "My SSN is 123-45-6789",
    policies=["pii-detection"]
)

result.passed       # False
result.violations   # [Violation(policy_id="pii-detection", pattern="ssn", ...)]

@guard.protect()

Decorator for function-level enforcement.

from overrule import Guard, PolicyAction

guard = Guard()

@guard.protect(policies=["injection-detection"], action=PolicyAction.BLOCK)
async def query_database(sql: str) -> str:
    return await db.execute(sql)

guard.register_policy()

Register custom policies.

from overrule.policies.base import BasePolicy, PolicyResult
from overrule.models.violation import Violation

class TopicRestriction(BasePolicy):
    policy_id = "topic-restriction"

    def evaluate(self, content: str, *, direction: str = "input") -> PolicyResult:
        if "medical advice" in content.lower():
            return PolicyResult(
                passed=False,
                violations=[Violation(
                    policy_id=self.policy_id,
                    severity="high",
                    description="Medical advice is restricted",
                )],
            )
        return PolicyResult(passed=True, violations=[])

guard.register_policy(TopicRestriction)

Built-in Policies

Policy ID	What It Detects
pii-detection	Credit cards, SSN, email, phone, IBAN, passport numbers, IPv4 addresses
injection-detection	8 prompt injection patterns + 5 SQL injection patterns
toxicity-detection	Profanity, slurs, hate speech, violence incitement, dangerous instructions

Policy Actions

Action	Behavior	Default?
PolicyAction.WARN	Detect violations, surface in response.violations, continue execution	Yes
PolicyAction.BLOCK	Raise ViolationError, halt execution — LLM never called on input violations
PolicyAction.REDACT	Replace matched content with [POLICY_ID] tokens in output
PolicyAction.LOG	Record violation silently, continue execution (telemetry only)

Important: Prompt injection violations always block regardless of default_action (they set violation.blocked=True). To block on all policy violations, set default_action=PolicyAction.BLOCK.

Integrations

LangChain

from overrule.integrations import OverruleCallback

callback = OverruleCallback(
    policies=["pii-detection", "injection-detection", "toxicity-detection"],
    action=PolicyAction.BLOCK,
    on_violation=lambda v: alert_team(v),  # optional hook
)

# Drop into any LangChain LLM
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o", callbacks=[callback])
result = llm.invoke("Hello world")  # automatically governed

---

Performance

Metric	Value
Policy evaluation	<1ms typical, ~30ms on 100KB inputs
Network calls on hot path	0
Buffer capacity	10,000 events
Flush interval	5s (configurable)
Test suite	140 tests passing
Python versions	3.10 · 3.11 · 3.12 · 3.13 · 3.14

---

Security Model

Enforcement Behavior

Out of the box, Overrule operates in WARN mode: violations are detected, logged to telemetry, and surfaced in response.violations — but the LLM call proceeds. This lets you integrate safely without breaking existing flows.

Exception: Prompt injection violations always block, regardless of default_action. They set violation.blocked=True, which triggers ViolationError before the LLM is called.

To enforce hard blocking on all violations:

from overrule import Guard, PolicyAction

guard = Guard(default_action=PolicyAction.BLOCK)

Fail-Open Architecture

By default, fail_open=True: if the SDK crashes during policy evaluation, the LLM call proceeds unguarded rather than failing your application. This is a deliberate availability-over-security tradeoff.

For strict enforcement where governance failure = application failure:

guard = Guard(fail_open=False)

Streaming Limitations

guard.stream() evaluates policies incrementally (every N chunks). Tokens yielded before a violation is detected cannot be recalled. For strict enforcement where no violating content may reach the user, use non-streaming guard.chat() with default_action=BLOCK.

Content Scanning

Large inputs (>100KB) are sampled: head, middle, and tail windows are scanned. A WARNING log is emitted when truncation occurs. The sampling strategy covers all three regions to prevent evasion by placing payloads in the center.

Other Protections

• API keys never exposed in repr(), str(), or serialized output
• PII redaction shows only last 4 characters (no BIN/prefix leakage)
• Config values are bounds-validated (batch_size, flush_interval, etc.)
• PEP 561 compliant (py.typed marker for downstream type checking)
• No secrets in logs — all sensitive values masked in debug output
• Policy timeout (5s) prevents ReDoS from triggering fail-open bypass

---

Cloud Dashboard

The Overrule cloud dashboard at overrule.dev provides:

Feature	Description
Posture Score	At-a-glance governance health metric
Event Stream	Filterable, paginated log of every governed LLM call
Policy Metrics	Effectiveness rates, violation counts, status per policy
API Key Management	Create, revoke, usage tracking — plan-gated limits
Billing	Subscription management with usage metering
Settings	Webhook configuration, profile, account management

Plans

	Free	Growth	Scale	Enterprise
Events/month	10,000	1,000,000	10,000,000	Unlimited
API keys	5	25	100	Unlimited
Webhooks	1	5	20	Unlimited
Rate limit	200/min	2,000/min	10,000/min	50,000/min
Retention	7 days	30 days	90 days	365 days
Price	Free	$499/mo	$2,999/mo	Custom

---

Project Structure

overrule-sdk/
├── overrule/
│   ├── __init__.py              # Public API (Guard, SyncGuard, PolicyAction, policies)
│   ├── guard.py                 # Core Guard class (async context manager, REDACT flow, streaming)
│   ├── stream.py                # StreamGuard — token-by-token policy eval for streaming
│   ├── sync.py                  # SyncGuard wrapper (background thread + event loop)
│   ├── exceptions.py            # Exception hierarchy (ViolationError, TransportError, etc.)
│   ├── logging.py               # Structured logging utilities
│   ├── integrations/
│   │   ├── __init__.py          # Framework integration exports
│   │   └── langchain.py         # OverruleCallback for LangChain
│   ├── models/
│   │   ├── config.py            # GuardConfig + PolicyAction enum (BLOCK, LOG, WARN, REDACT)
│   │   ├── event.py             # InterceptEvent (structured governance event)
│   │   └── violation.py         # Violation model (policy_id, severity, direction)
│   ├── policies/
│   │   ├── base.py              # BasePolicy abstract class + PolicyResult
│   │   ├── registry.py          # Thread-safe PolicyRegistry
│   │   ├── pii.py               # PII detection (7 patterns, raw_match metadata)
│   │   ├── injection.py         # Prompt injection (8) + SQL injection (5) patterns
│   │   └── toxicity.py          # Toxicity detection (profanity, slurs, violence, 3 tiers)
│   └── transport/
│       ├── reporter.py          # Async EventReporter (batching, backoff, circuit breaker)
│       └── dead_letter.py       # Dead-letter queue (persist dropped events to disk)
├── tests/                       # 137 tests (pytest)
├── examples/                    # Runnable integration examples
├── pyproject.toml               # Build config + dependencies
├── CHANGELOG.md                 # Version history
└── LICENSE                      # MIT

---

Compliance Mapping

EU AI Act Requirement	Overrule Implementation
Art. 13 — Transparency & logging	Every LLM call logged with model, tokens, latency, policies, violations
Art. 14 — Human oversight	Dashboard shows real-time enforcement stream, violation alerts
Art. 15 — Accuracy & robustness	Policy enforcement prevents degraded/adversarial outputs
Audit evidence	Structured event export for regulators
Enforcement date	August 2, 2026 — fines up to €35M / 7% global revenue

---

Roadmap

• Core Guard with fail-open architecture
• PII detection policy (credit cards, SSN, email, phone, IBAN, passport, IPv4)
• Injection detection policy (8 prompt injection + 5 SQL injection patterns)
• Async + Sync APIs (Guard + SyncGuard)
• Multi-provider support (OpenAI + Anthropic)
• Custom policy engine (BasePolicy interface)
• Decorator API (@guard.protect())
• Standalone evaluation (guard.evaluate())
• Circuit breaker (5 failures → open → 30s cooldown → recovery)
• Bounded event buffer (10K max, graceful shutdown flush)
• Exponential backoff with jitter on transport failures
• Cloud event streaming (POST /api/v1/events)
• Environment-based configuration
• Published on PyPI (pip install overrule)
• Toxicity detection policy (profanity, slurs, violence, 3 severity tiers)
• REDACT action (replace violations with tokens instead of blocking)
• Output policy enforcement (response scanning)
• Streaming interception (guard.stream() with incremental evaluation)
• LangChain integration (OverruleCallback drop-in handler)
• Dead-letter queue (failed events persisted to disk, auto-recovered)
• Policy hot-reload (update policies at runtime without restart)
• Credit card detection with dash/space formats
• 137-test suite (pytest)
• PEP 561 compliant (py.typed)
• CrewAI integration (agent-level governance)
• OpenAI Agents SDK wrapper
• Rust core for <100μs evaluation
• Policy marketplace (community-contributed policies)

---

Examples

The examples/ directory contains runnable scripts for common use cases:

Example	Description	Requires LLM Key
quickstart.py	Full integration test — LLM call + PII + injection	Yes
evaluate_only.py	Policy evaluation without LLM calls	No
custom_policy.py	Build your own policy (topic restriction, length limits)	No
sync_usage.py	Synchronous API for scripts and notebooks	No

# Run any example
cd overrule-sdk
export OVERRULE_API_KEY=sk_ovr_...
python examples/evaluate_only.py

---

Development

# Clone
git clone https://github.com/overruledev/overrule-sdk.git
cd overrule-sdk

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest --cov=overrule --cov-fail-under=80

# Lint + format check
ruff check .
ruff format --check .

# Type check (strict)
mypy overrule/

CI/CD Pipeline

Every push and PR triggers a production-grade CI pipeline:

Stage	What It Does
Lint	ruff check + ruff format --check
Type Check	mypy in strict mode
Security Audit	pip-audit scans all dependencies for known vulnerabilities
Test	pytest across Python 3.10–3.13 with 80% coverage gate
Build & Verify	Builds sdist + wheel, twine check, install verification, 500KB size cap

PR Quality Gates (run on pull requests only):

• New dependency detection with review notice
• Debug print() statement detection
• TODO/FIXME/HACK tracker
• Secret pattern scanning (hard fail)
• .env file leak detection (hard fail)
• Version bump notification

---

Contributing

We're building in public. Contributions welcome.

# Fork + clone
git clone https://github.com/yourusername/overrule-sdk.git

# Create feature branch
git checkout -b feature/your-feature

# Make changes, then run the full CI suite locally
pytest --cov=overrule --cov-fail-under=80   # Tests + coverage
ruff check .                                 # Lint
ruff format --check .                        # Format
mypy overrule/                               # Type check

git commit -m "feat: your feature description"
git push origin feature/your-feature

All PRs must pass lint, typecheck, security audit, tests (80%+ coverage), and build verification before merge.

---

Contact

Purpose	Email
General inquiries	hello@overrule.dev
Customer support	support@overrule.dev
Enterprise sales	sales@overrule.dev
Founder	founders@overrule.dev

---

License

MIT License. See LICENSE for details.

---

Built for teams shipping AI to production.
_{Overrule — because governance shouldn't slow you down.}