NVIDIA NeMo Guardrails Library
Documentation · Quickstart · Use cases · Integrations · Guardrail catalog · Examples · Community · Security · Releases · Changelog
NeMo Guardrails is an open-source library for applying programmable safety and security controls to LLM and agentic applications. Agents run commands, call APIs, query your data, and read the web. Conversational applications apply input and output checks; retrieval-augmented generation (RAG) systems filter retrieved content and check answers against evidence.
You use the NeMo Guardrails library to declare what each agent may do in a policy, combining low-latency local rules with model-backed checks for jailbreaks, injected instructions, and unsafe tool calls. Your application or agent harness runs these checks and uses their results to decide whether to continue.
pip install nemoguardrails
Requires Python 3.11–3.13. Refer to the installation guide for optional packages and setup.
What You Can Build
- AI agents and coding assistants: Check proposed tool calls and returned results with tool rails, or connect an agent through the hooks server.
- Conversational applications: Apply input and output checks.
- Retrieval-augmented generation (RAG): Detect or mask sensitive data in retrieved content and check whether answers are grounded in evidence.
How It Works
Use the NVIDIA NeMo Guardrails library to define your policy in configuration files that specify the rails, models, and prompts to use. Run the library in your Python application or deploy it as a server. Choose check mode or proxy mode based on how your application manages model requests.
Check Mode
Your application or agent harness calls the LLM and executes tools directly. At each point that requires a policy decision, send the relevant prompt, model response, tool call, or tool result to the library for checking. Use the result to handle blocked or modified content before you execute a tool, return a response, or pass a tool result to the model.
Use check_async() in Python, POST /v1/checks on the Guardrails server, or the hooks server for coding agents.
For tool checks with check_async(), use the IORails engine and specify RailType.TOOL_CALL or RailType.TOOL_RESULT in rail_types.
Check tool calls before execution and tool results after execution.
Proxy Mode
Route model requests through the library to apply the configured input and output rails. Input rails can block a request before it reaches the application LLM. For non-streaming requests, output rails check the response before the library returns it. Blocked content produces a refusal or an error, depending on the integration and configuration.
Use generate_async() or stream_async() in Python, or POST /v1/chat/completions on the Guardrails server.
Your application executes tools; supported integrations can check tool calls in model responses and tool results in subsequent requests.
The gateway server also supports POST /v1/chat/completions and POST /v1/responses, but its built-in profiles do not support tool content.
Each gateway /v1/responses request must include "store": false.
When output streaming is enabled for stream_async(), the default stream_first: true setting delivers chunks before checking them.
To check each chunk before delivery, set rails.output.streaming.stream_first: false.
For configuration details, refer to output rail streaming.
Latency depends on the configured rules, models, and services. Rail support varies by engine and integration. Refer to rail engine support, configuration, and Choose Your Integration.
Why NeMo Guardrails
- Govern what agents do. Validate tool names and argument schemas before execution. Check tool results and apply per-tool policies before returning them to the model.
- Keep your model and agent loop. Use checks without main-model generation to add guardrails to an existing application. Model-backed checks still call their configured model or service.
- Compose the policy you need. Combine local regex and CEL rules with safety models and third-party checks.
- Connect your runtime. Choose an integration for your application, shared service, or coding agent.
- Open and yours to run. The library is Apache 2.0 licensed. Choose supported model providers and run checks with local detectors or hosted inference endpoints.
Quickstart
Check a support agent's proposed refund against a $500 limit, without a model or API key. Install the optional CEL extra for this example:
pip install "nemoguardrails[cel]"
The example validates the tool's argument schema and blocks calls over the limit. Expand it and save it as check_refund.py:
Complete Python example
import asyncio
import json
from nemoguardrails import Guardrails, RailsConfig
from nemoguardrails.rails.llm.options import RailStatus, RailType
config = RailsConfig.from_content(yaml_content="""
models: []
rails:
tool_call:
flows: [tool call validation]
per_tool:
issue_refund: [cel check tool call]
config:
cel:
tool_call:
issue_refund:
expressions: ["args.amount > 500"]
""")
TOOLS = [{
"type": "function",
"function": {
"name": "issue_refund",
"parameters": {
"type": "object",
"properties": {"amount": {"type": "number", "minimum": 0}},
"required": ["amount"],
"additionalProperties": False,
},
},
}]
async def main():
async with Guardrails(config, require_iorails=True) as rails:
for amount in (50, 5000):
call = {
"role": "assistant",
"tool_calls": [{
"id": "call_1",
"type": "function",
"function": {
"name": "issue_refund",
"arguments": json.dumps({"amount": amount}),
},
}],
}
result = await rails.check_async(
[call], rail_types=[RailType.TOOL_CALL], tools=TOOLS
)
print(f"${amount}: {result.status.value}")
if result.status == RailStatus.PASSED:
print("Ready for the application to execute")
else:
print(f"Rail: {result.rail}")
print(f"Reason: {result.reason}")
asyncio.run(main())
Run it from the directory where you saved the file:
python check_refund.py
Expected output:
$50: passed
Ready for the application to execute
$5000: blocked
Rail: cel check tool call
Reason: cel check tool call
The example does not issue refunds. Your application executes a tool only after its checks pass. For more configuration and error-handling examples, refer to tool calling and checking messages.
Choose Your Integration
Connect NeMo Guardrails where your application or agent can enforce its checks.
| What are you building? | How to connect |
|---|---|
| A shared guardrails service for multiple applications or teams | Use the Checks API (/v1/checks) to request verdicts while each application manages its model and execution loop. |
| A guardrails layer for existing model APIs | Use the gateway server to check supported requests and responses while preserving provider formats. |
| A coding assistant or agent harness | Use native hooks or NeMo Relay to check tool calls before execution. |
| An agent platform, framework integration, or custom loop | Embed the Python SDK, call the Checks API, or use a supported framework integration. |
Guardrail Catalog
Choose checks for your policy and workflow:
| Check | Purpose |
|---|---|
| Content safety | Check inputs and outputs against safety policies. |
| Topic control | Check inputs against configured topical guidelines. |
| Jailbreak detection | Detect jailbreak attempts with configured heuristics or classifiers. |
| Injection detection | Check model output for code, SQL, template, and cross-site scripting patterns. |
| Sensitive data and PII | Detect or mask sensitive data in inputs, outputs, and retrieved content. |
| Fact-checking and hallucination checks | Check answers against evidence or compare model responses for consistency. |
| Tool-call and tool-result validation | Validate declared tools, argument schemas, and tool-result linkage and structure. |
| Tool safety checks | Evaluate tool calls and results against per-tool safety instructions. |
| Regex and CEL | Apply deterministic pattern and expression rules. |
| LLM self-checks | Use configured LLM prompts for input moderation, output moderation, or fact-checking. |
| Language detection | Restrict input to configured languages. |
| Third-party integrations | Connect managed safety and moderation services. |
NVIDIA Nemotron Safety models include Nemotron 3.5 Content Safety, Nemotron Content Safety Reasoning 4B, and Llama 3.1 Nemotron Safety Guard 8B V3. Follow the linked examples and guides for model-specific setup.
Use the evaluation tools to assess your policies and garak to scan LLM applications for vulnerabilities. Refer to the vulnerability-scanning guide for an example.
Benchmarking Guardrails Performance
The following numbers show the latency that the Guardrails API server adds to each request, measured on a 14-inch MacBook Pro with an Apple M4 (10 cores, 24 GB).
The server runs the content_safety_local configuration, which checks each request with an input rail and an output rail.
Mock LLMs replace the main model and the content-safety model with fixed latencies, so each request spends 5,000 ms in model calls: 500 ms for each content-safety check and 4,000 ms for the main model.
AIPerf sends non-streaming requests for 180 seconds at each concurrency level, with prompts of about 100 input tokens.
The following chart shows the added latency at concurrency 64, where each LLMRails label also gives its multiple of the IORails value.
| Concurrency | IORails p50 (ms) | IORails p90 (ms) | IORails p99 (ms) | LLMRails p50 (ms) | LLMRails p90 (ms) | LLMRails p99 (ms) |
|---|---|---|---|---|---|---|
| 1 | 11 | 13 | 30 | 39 | 40 | 53 |
| 2 | 13 | 15 | 20 | 62 | 70 | 104 |
| 4 | 13 | 15 | 19 | 85 | 100 | 127 |
| 8 | 13 | 19 | 68 | 136 | 179 | 208 |
| 16 | 15 | 21 | 30 | 97 | 289 | 416 |
| 32 | 13 | 19 | 106 | 88 | 274 | 860 |
| 64 | 13 | 22 | 126 | 93 | 669 | 2,094 |
| 128 | 17 | 774 | 2,880 | 430 | 1,422 | 5,182 |
Each value is the request-latency percentile minus the 5,000 ms of model calls. It includes the server's three HTTP calls to the mock models, about 2.5 ms each.
To reproduce these numbers, follow the benchmarking guide with two changes:
- Set
benchmark_duration: 180in the AIPerf config; the shipped configs use 60 seconds. - For the IORails columns, start the server with
NEMO_GUARDRAILS_IORAILS_ENGINE=true; the guide's Procfile starts the server without it, so the server runs LLMRails by default.
The same tooling can be used to benchmark production Guardrails deployments using real models instead of mock LLMs. For this use-case the Guardrails configuration should point to production LLMs, and AIPerf should run with representative production datasets.
Community Usage
This anonymous usage snapshot covers 0.22.0 and 0.23.0 release builds from May 22 to August 18, 2026, with an estimated 665,798 distinct startup sessions.
View rail usage, feature usage, and configured rail counts
Session counts are estimates, and a session can use multiple rail types or features. Refer to the aggregate data and methodology for definitions and counting limits.
Telemetry and Privacy
The library sends anonymous usage telemetry that excludes prompts, completions, model names, credentials, and endpoints. Set NEMO_GUARDRAILS_NO_USAGE_STATS=1 to opt out. This does not disable configured model or guardrail service calls, which can send content to those endpoints. Refer to telemetry and privacy for details and other opt-out options.
Third-party endpoints have separate terms and privacy practices. The NVIDIA NeMo Guardrails library can use inference endpoints such as NVIDIA Build (build.nvidia.com). If you use NVIDIA Build or another third-party endpoint, that endpoint's terms of service and privacy practices apply independently of the library. Any telemetry opt-out in the NVIDIA NeMo Guardrails library does not extend to the endpoint you choose. NVIDIA Build is intended for evaluation and testing only and must not be used in production environments. Do not submit confidential information or personal data when using NVIDIA Build.
Contributing and Support
Refer to CONTRIBUTING.md and AI_POLICY.md for contribution requirements. Ask usage questions in GitHub Discussions, and follow SECURITY.md to report vulnerabilities.
The library is licensed under the Apache License, Version 2.0. If you use it in research, cite the EMNLP 2023 paper.
Metadata
Release files for nemoguardrails 0.25.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| nemoguardrails-0.25.0.tar.gz | 1.1 MB | With provenance | Details for nemoguardrails-0.25.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| nemoguardrails-0.25.0-py3-none-any.whl | Python 3 | none | any | Details for nemoguardrails-0.25.0-py3-none-any.whl |
Total release size: 2.7 MB
Release files / nemoguardrails-0.25.0.tar.gz
| Download URL | nemoguardrails-0.25.0.tar.gz |
|---|---|
| Size | 1.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2d22dc2ba43df5ea42bb605ed663ce0fa1a1e57df169c7e683556d72021775ab
|
|
BLAKE2b-256 checksum How to use checksums |
2c24ca3120a6ff7eeebb1ae6e88bf33868019a567501e7efa9dec93875f1b0d7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.
Transparency logRelease files / nemoguardrails-0.25.0-py3-none-any.whl
| Download URL | nemoguardrails-0.25.0-py3-none-any.whl |
|---|---|
| Size | 1.6 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a138e8770d2d8889d4c0c0d989dc9801a3687e285924f5fa6a4ff0c8c8b6100e
|
|
BLAKE2b-256 checksum How to use checksums |
249770114a5c3e8a0431c3c7c0047727703a9b4d40f4d5c71ea50ea2321e1e9f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.
Transparency logSecurity for nemoguardrails 0.25.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for nemoguardrails-0.25.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 19, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.