Railproof

The short version

AI agents can now send emails, move money, call APIs, modify tickets, and touch production systems.

The problem is not whether the model can choose a tool. The problem is whether that exact action should be allowed.

Railproof puts a deterministic security contract in front of every tool call. It checks the tool, complete arguments, user, tenant, roles, data labels, provenance, approvals, and session budgets before application code runs.

If the action is not explicitly allowed, it does not execute.

Why product teams need this

Most guardrails answer one of these questions:

• Is the request safe to discuss?
• Does the tool call match a JSON shape?
• Is this tool on an allowlist?

Those checks are useful, but they do not answer the business question:

“Should this agent be allowed to perform this exact operation for this user, with this data, in this session, right now?”

A request can be perfectly valid JSON and still be wrong. It can send data to the wrong host, use retrieved content as an email recipient, exceed a spending budget, or reuse an approval after the action changes.

Railproof is the enforcement layer for that decision.

What this repository covers

This repository contains the complete v0.1.0 alpha implementation:

What this means for a product owner

You define the rules once. Every connected agent and tool call goes through the same decision contract.

1. Stop actions that are technically valid but business-invalid

JSON Schema can confirm that an email has a to field and a body. Railproof can additionally say:

• never send sensitive data to an external sink
• never let retrieved content choose an email recipient
• only call a transfer tool within a session budget
• allow weather lookups but require approval for external email
• deny unknown tools and unmatched actions

The policy does not ask the model whether the action is safe. The model has no authority to override it.

2. Require approval for high-impact actions

A rule can stop an action and return an approval challenge. The approval is cryptographically bound to the exact action.

Change the recipient, amount, tool, user, or policy and the old approval no longer works. Reuse the same approval and it is rejected.

The v0.1.0 broker is the security primitive, not a finished approval dashboard or human workflow. Your product can connect it to the approval system you already use.

3. Enforce budgets across a session

Policies can limit:

• how many times a tool is called
• how much numeric value accumulates across calls
• per-session budgets that can be applied alongside user and tenant rules

The reservation happens before execution and is atomic for concurrent calls. A burst of parallel requests cannot spend the same remaining budget twice.

4. Keep a useful decision trail

Railproof records whether a call was allowed, denied, waiting for approval, started, completed, or failed.

Built-in evidence includes identifiers, hashes, matched rules, reasons, outcomes, and a hashed principal. It does not store raw arguments, approval tokens, prompts, or tool results.

Use the in-memory sink for local work or the append-only JSONL sink for a file-based audit trail.

5. Give different agent stacks the same security contract

The policy engine is protocol-independent. The repository includes adapters that normalize:

• OpenAI Chat Completions function calls
• MCP JSON-RPC 2.0 tools/call requests

Both become the same canonical event before policy evaluation.

6. Test policy behavior before production

The CLI can validate policies, evaluate one event, and replay JSONL decision fixtures. That gives product and security teams a way to review expected outcomes without making real tool calls.

The repository also includes a benchmark and its raw result. Railproof scored 10/10 against the benchmark’s declared requirements. NeMo Guardrails 0.24.0’s built-in tool-call validator scored 5/10.

That is a narrow comparison against one NeMo rail. It is not a claim that Railproof replaces NeMo’s conversational rails, model-backed detectors, server, or observability.

How a decision works

Agent proposes a tool call
          |
          v
Adapter or GuardedTools creates one canonical action snapshot
          |
          v
Policy validates tool, arguments, identity, labels, provenance, and session limits
          |
       +--+-------------------+
       |                      |
     deny             allow or approval
       |                      |
   no execution       reserve limits and verify approval
                              |
                              v
                    wrapped executor runs exact snapshot
                              |
                              v
                    evidence records the outcome

The important boundary is simple: application code does not receive a tool result until the action has passed the contract.

Feature detail

Strict policy compiler

Policies use the railproof.dev/v1alpha1 format. Compilation fails closed when the policy contains:

• unknown fields or duplicate keys
• an unsafe default
• undeclared tools or limits
• unsupported stages, effects, operators, or fields
• invalid JSON Schema
• schema references not supported by this alpha
• oversized policy input or excessive YAML aliases

Every compiled policy receives a deterministic SHA-256 policy hash.

Deterministic authorization

The decision engine evaluates locally with no model call and no network call. It supports:

• equality and inequality
• membership and exclusion
• numeric upper and lower bounds
• field labels such as sensitive
• provenance such as retrieved.email
• principal ID, tenant, and roles
• tool risk and sink metadata
• per-session call counts and numeric sums

Decision precedence is fixed:

deny > require approval > allow > unmatched deny

There is no implicit allow.

Exact-action binding

Railproof canonicalizes the caller’s arguments once into isolated event and execution snapshots. The hash covers:

• policy
• session
• principal identity, tenant, and roles
• tool name
• complete arguments
• labels
• provenance

This prevents a mutable or stateful caller mapping from being authorized as one action and executed as another.

Approval tokens

The built-in approval broker uses HMAC-SHA-256 and requires a signing key of at least 32 bytes. Tokens bind the action hash, policy hash, principal, approver, expiry, and random nonce.

It rejects forged, modified, malformed, expired, cross-principal, and reused tokens. Token replay protection is process-local in v0.1.0.

Safe execution wrapper

GuardedTools combines authorization and execution so application code does not need to remember a separate “check first, execute later” rule.

It supports synchronous and asynchronous executors, prevents denied executor entry, reserves budgets atomically, bounds active session state, and refuses session teardown while an execution is in flight.

Evidence sinks

Two sinks are included:

• InMemoryEvidenceSink with bounded retention, defaulting to 10,000 records
• JsonlEvidenceSink for append-only newline-delimited records

The built-in record contains event/session/tool identifiers, a principal hash, action/policy/decision hashes, outcome, matched rules, and reason codes.

CLI and replay

railproof validate policy.yaml
railproof check policy.yaml event.json
railproof replay policy.yaml fixtures.jsonl

validate compiles a policy and prints its hash. check evaluates one event. replay compares actual outcomes with expected outcomes and returns a failing exit code when fixtures disagree.

Install

Once the first release is published, install Railproof from PyPI:

python -m pip install railproof

Or with uv:

uv add railproof

The package includes the runtime library and the railproof command. No repository clone is needed to use it.

Quick start from source

Requires Python 3.11+ and uv.

git clone https://github.com/ppradyoth/railproof.git
cd railproof
uv sync --group dev --group test
uv run railproof validate examples/policy.yaml
uv run python examples/demo.py

The demo shows a benign weather call, a denied sensitive email, an approval challenge, and an approved email.

Add it to an application

Register the real executor once, then route calls through GuardedTools:

from railproof.executor import GuardedTools, ToolRegistry
from railproof.models import Principal
from railproof.policy import load_policy

policy = load_policy("policy.yaml")
registry = ToolRegistry()
registry.register("get_weather", get_weather)

tools = GuardedTools(policy, registry)
result = await tools.call(
    "get_weather",
    {"city": "Bengaluru"},
    principal=Principal(id="user-123", tenant="tenant-a"),
    session_id="session-123",
)

For high-impact actions, configure an ApprovalBroker, catch ApprovalRequired, obtain approval through your application workflow, and call the same action with the returned token.

See the complete example policy and runnable demo.

What Railproof is not

This alpha is an enforcement runtime, not a complete AI platform. It does not provide:

• a model or content-moderation system
• a sandbox against code that bypasses the wrapper
• automatic taint tracking or provenance attestation
• tool-result validation after execution
• argument rewriting or policy-driven mutation
• persistent or distributed session budgets and nonce storage
• a hosted control plane, dashboard, or approval UI
• a TypeScript SDK

These are explicit boundaries. Production teams should keep the underlying executors private, default policies to deny, protect signing keys, and treat labels and provenance as application-attested inputs.

Evidence for the v0.1.0 alpha

The benchmark uses NeMo Guardrails 0.24.0’s real deterministic ToolCallRailAction._validate implementation with no model or network calls. Read the benchmark method before interpreting the comparison.

Repository map

Development

uv run ruff format --check .
uv run ruff check .
uv run ty check src tests examples
uv run pytest -q
uv run pip-audit
uv build

Read SECURITY.md before production use. Contributions should include an adversarial fixture, a benign control, an expected decision, and proof of whether the executor was reached. See CONTRIBUTING.md.

Documentation

• Architecture
• Product specification
• Threat model
• NeMo comparison
• Benchmark method
• Project plan
• Roadmap

Apache-2.0 licensed.