Skip to main content

NVIDIA NeMo Guardrails Library

License PyPI Python Tests/Linux Tests/Windows Tests/macOS Lint Documentation Total downloads Monthly downloads

Documentation · Quickstart · Use cases · Integrations · Guardrail catalog · Examples · Community · Security · Releases · Changelog

NeMo Guardrails is an open-source library for applying programmable safety and security controls to LLM and agentic applications. Agents run commands, call APIs, query your data, and read the web. Conversational applications apply input and output checks; retrieval-augmented generation (RAG) systems filter retrieved content and check answers against evidence.

You use the NeMo Guardrails library to declare what each agent may do in a policy, combining low-latency local rules with model-backed checks for jailbreaks, injected instructions, and unsafe tool calls. Your application or agent harness runs these checks and uses their results to decide whether to continue.

pip install nemoguardrails

Requires Python 3.11–3.13. Refer to the installation guide for optional packages and setup.

What You Can Build

How It Works

Use the NVIDIA NeMo Guardrails library to define your policy in configuration files that specify the rails, models, and prompts to use. Run the library in your Python application or deploy it as a server. Choose check mode or proxy mode based on how your application manages model requests.

Check Mode

Your application or agent harness calls the LLM and executes tools directly. At each point that requires a policy decision, send the relevant prompt, model response, tool call, or tool result to the library for checking. Use the result to handle blocked or modified content before you execute a tool, return a response, or pass a tool result to the model.

Use check_async() in Python, POST /v1/checks on the Guardrails server, or the hooks server for coding agents. For tool checks with check_async(), use the IORails engine and specify RailType.TOOL_CALL or RailType.TOOL_RESULT in rail_types. Check tool calls before execution and tool results after execution.

Check mode: your application calls the LLM and tools directly and requests checks through the Python SDK, Guardrails server, or hooks server. Configured rails return a result and can call a guardrail model.

Proxy Mode

Route model requests through the library to apply the configured input and output rails. Input rails can block a request before it reaches the application LLM. For non-streaming requests, output rails check the response before the library returns it. Blocked content produces a refusal or an error, depending on the integration and configuration.

Use generate_async() or stream_async() in Python, or POST /v1/chat/completions on the Guardrails server. Your application executes tools; supported integrations can check tool calls in model responses and tool results in subsequent requests. The gateway server also supports POST /v1/chat/completions and POST /v1/responses, but its built-in profiles do not support tool content. Each gateway /v1/responses request must include "store": false.

When output streaming is enabled for stream_async(), the default stream_first: true setting delivers chunks before checking them. To check each chunk before delivery, set rails.output.streaming.stream_first: false. For configuration details, refer to output rail streaming.

Proxy mode: your application routes model requests through the library, which applies configured rails and forwards allowed requests to the LLM. Rails can call a guardrail model. Your application executes tools.

Latency depends on the configured rules, models, and services. Rail support varies by engine and integration. Refer to rail engine support, configuration, and Choose Your Integration.

Why NeMo Guardrails

  • Govern what agents do. Validate tool names and argument schemas before execution. Check tool results and apply per-tool policies before returning them to the model.
  • Keep your model and agent loop. Use checks without main-model generation to add guardrails to an existing application. Model-backed checks still call their configured model or service.
  • Compose the policy you need. Combine local regex and CEL rules with safety models and third-party checks.
  • Connect your runtime. Choose an integration for your application, shared service, or coding agent.
  • Open and yours to run. The library is Apache 2.0 licensed. Choose supported model providers and run checks with local detectors or hosted inference endpoints.

Quickstart

Check a support agent's proposed refund against a $500 limit, without a model or API key. Install the optional CEL extra for this example:

pip install "nemoguardrails[cel]"

The example validates the tool's argument schema and blocks calls over the limit. Expand it and save it as check_refund.py:

Complete Python example
import asyncio
import json

from nemoguardrails import Guardrails, RailsConfig
from nemoguardrails.rails.llm.options import RailStatus, RailType

config = RailsConfig.from_content(yaml_content="""
models: []
rails:
  tool_call:
    flows: [tool call validation]
    per_tool:
      issue_refund: [cel check tool call]
  config:
    cel:
      tool_call:
        issue_refund:
          expressions: ["args.amount > 500"]
""")

TOOLS = [{
    "type": "function",
    "function": {
        "name": "issue_refund",
        "parameters": {
            "type": "object",
            "properties": {"amount": {"type": "number", "minimum": 0}},
            "required": ["amount"],
            "additionalProperties": False,
        },
    },
}]


async def main():
    async with Guardrails(config, require_iorails=True) as rails:
        for amount in (50, 5000):
            call = {
                "role": "assistant",
                "tool_calls": [{
                    "id": "call_1",
                    "type": "function",
                    "function": {
                        "name": "issue_refund",
                        "arguments": json.dumps({"amount": amount}),
                    },
                }],
            }
            result = await rails.check_async(
                [call], rail_types=[RailType.TOOL_CALL], tools=TOOLS
            )
            print(f"${amount}: {result.status.value}")
            if result.status == RailStatus.PASSED:
                print("Ready for the application to execute")
            else:
                print(f"Rail: {result.rail}")
                print(f"Reason: {result.reason}")


asyncio.run(main())

Run it from the directory where you saved the file:

python check_refund.py

Expected output:

$50: passed
Ready for the application to execute
$5000: blocked
Rail: cel check tool call
Reason: cel check tool call

The example does not issue refunds. Your application executes a tool only after its checks pass. For more configuration and error-handling examples, refer to tool calling and checking messages.

Choose Your Integration

Connect NeMo Guardrails where your application or agent can enforce its checks.

What are you building? How to connect
A shared guardrails service for multiple applications or teams Use the Checks API (/v1/checks) to request verdicts while each application manages its model and execution loop.
A guardrails layer for existing model APIs Use the gateway server to check supported requests and responses while preserving provider formats.
A coding assistant or agent harness Use native hooks or NeMo Relay to check tool calls before execution.
An agent platform, framework integration, or custom loop Embed the Python SDK, call the Checks API, or use a supported framework integration.

Guardrail Catalog

Choose checks for your policy and workflow:

Check Purpose
Content safety Check inputs and outputs against safety policies.
Topic control Check inputs against configured topical guidelines.
Jailbreak detection Detect jailbreak attempts with configured heuristics or classifiers.
Injection detection Check model output for code, SQL, template, and cross-site scripting patterns.
Sensitive data and PII Detect or mask sensitive data in inputs, outputs, and retrieved content.
Fact-checking and hallucination checks Check answers against evidence or compare model responses for consistency.
Tool-call and tool-result validation Validate declared tools, argument schemas, and tool-result linkage and structure.
Tool safety checks Evaluate tool calls and results against per-tool safety instructions.
Regex and CEL Apply deterministic pattern and expression rules.
LLM self-checks Use configured LLM prompts for input moderation, output moderation, or fact-checking.
Language detection Restrict input to configured languages.
Third-party integrations Connect managed safety and moderation services.

NVIDIA Nemotron Safety models include Nemotron 3.5 Content Safety, Nemotron Content Safety Reasoning 4B, and Llama 3.1 Nemotron Safety Guard 8B V3. Follow the linked examples and guides for model-specific setup.

Use the evaluation tools to assess your policies and garak to scan LLM applications for vulnerabilities. Refer to the vulnerability-scanning guide for an example.

Benchmarking Guardrails Performance

The following numbers show the latency that the Guardrails API server adds to each request, measured on a 14-inch MacBook Pro with an Apple M4 (10 cores, 24 GB). The server runs the content_safety_local configuration, which checks each request with an input rail and an output rail. Mock LLMs replace the main model and the content-safety model with fixed latencies, so each request spends 5,000 ms in model calls: 500 ms for each content-safety check and 4,000 ms for the main model. AIPerf sends non-streaming requests for 180 seconds at each concurrency level, with prompts of about 100 input tokens.

The following chart shows the added latency at concurrency 64, where each LLMRails label also gives its multiple of the IORails value.

Latency added by IORails and LLMRails at concurrency 64

Concurrency IORails p50 (ms) IORails p90 (ms) IORails p99 (ms) LLMRails p50 (ms) LLMRails p90 (ms) LLMRails p99 (ms)
1 11 13 30 39 40 53
2 13 15 20 62 70 104
4 13 15 19 85 100 127
8 13 19 68 136 179 208
16 15 21 30 97 289 416
32 13 19 106 88 274 860
64 13 22 126 93 669 2,094
128 17 774 2,880 430 1,422 5,182

Each value is the request-latency percentile minus the 5,000 ms of model calls. It includes the server's three HTTP calls to the mock models, about 2.5 ms each.

To reproduce these numbers, follow the benchmarking guide with two changes:

  • Set benchmark_duration: 180 in the AIPerf config; the shipped configs use 60 seconds.
  • For the IORails columns, start the server with NEMO_GUARDRAILS_IORAILS_ENGINE=true; the guide's Procfile starts the server without it, so the server runs LLMRails by default.

The same tooling can be used to benchmark production Guardrails deployments using real models instead of mock LLMs. For this use-case the Guardrails configuration should point to production LLMs, and AIPerf should run with representative production datasets.

Community Usage

This anonymous usage snapshot covers 0.22.0 and 0.23.0 release builds from May 22 to August 18, 2026, with an estimated 665,798 distinct startup sessions.

View rail usage, feature usage, and configured rail counts

Sessions by rail type

Sessions by library feature

Configured rails at startup

Session counts are estimates, and a session can use multiple rail types or features. Refer to the aggregate data and methodology for definitions and counting limits.

Telemetry and Privacy

The library sends anonymous usage telemetry that excludes prompts, completions, model names, credentials, and endpoints. Set NEMO_GUARDRAILS_NO_USAGE_STATS=1 to opt out. This does not disable configured model or guardrail service calls, which can send content to those endpoints. Refer to telemetry and privacy for details and other opt-out options.

Third-party endpoints have separate terms and privacy practices. The NVIDIA NeMo Guardrails library can use inference endpoints such as NVIDIA Build (build.nvidia.com). If you use NVIDIA Build or another third-party endpoint, that endpoint's terms of service and privacy practices apply independently of the library. Any telemetry opt-out in the NVIDIA NeMo Guardrails library does not extend to the endpoint you choose. NVIDIA Build is intended for evaluation and testing only and must not be used in production environments. Do not submit confidential information or personal data when using NVIDIA Build.

Contributing and Support

Refer to CONTRIBUTING.md and AI_POLICY.md for contribution requirements. Ask usage questions in GitHub Discussions, and follow SECURITY.md to report vulnerabilities.

The library is licensed under the Apache License, Version 2.0. If you use it in research, cite the EMNLP 2023 paper.

Metadata

Release files for nemoguardrails 0.25.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for nemoguardrails 0.25.0
File Size Uploaded Provenance Actions
nemoguardrails-0.25.0.tar.gz 1.1 MB With provenance Details for nemoguardrails-0.25.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for nemoguardrails 0.25.0
File Interpreter ABI Platform Actions
nemoguardrails-0.25.0-py3-none-any.whl Python 3 none any Details for nemoguardrails-0.25.0-py3-none-any.whl

Total release size: 2.7 MB

Release files / nemoguardrails-0.25.0.tar.gz

Download URL nemoguardrails-0.25.0.tar.gz
Size 1.1 MB
Tags Source
SHA-256 checksum
How to use checksums
2d22dc2ba43df5ea42bb605ed663ce0fa1a1e57df169c7e683556d72021775ab
BLAKE2b-256 checksum
How to use checksums
2c24ca3120a6ff7eeebb1ae6e88bf33868019a567501e7efa9dec93875f1b0d7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

Release files / nemoguardrails-0.25.0-py3-none-any.whl

Download URL nemoguardrails-0.25.0-py3-none-any.whl
Size 1.6 MB
Tags Python 3
SHA-256 checksum
How to use checksums
a138e8770d2d8889d4c0c0d989dc9801a3687e285924f5fa6a4ff0c8c8b6100e
BLAKE2b-256 checksum
How to use checksums
249770114a5c3e8a0431c3c7c0047727703a9b4d40f4d5c71ea50ea2321e1e9f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

Security for nemoguardrails 0.25.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for nemoguardrails-0.25.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.25.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page