PolicyAware AI Gateway & Agent Control Plane

PyPI: policyaware | Downloads: Pepy stats | Python: 3.10+ | License: Apache-2.0 | Docs: GitHub Pages

PolicyAware adds deny-by-default policy, PII redaction, MCP tool governance, model routing, runtime evaluation, local code scanning, and audit traces to LLM, RAG, and AI agent applications in minutes.

PolicyAware is an open-source AI control plane and security gateway for governed LLM applications, RAG pipelines, MCP-style tools, and autonomous AI agents. Distributed as a lightweight Python package (pip install policyaware), it helps teams inspect prompts, request context, tool calls, model routing decisions, outputs, local code, and audit traces before AI workflows move into production.

Unlike basic content filters that only check text strings, PolicyAware provides policy-aware governance across requests, tools, models, evaluations, and local code scans.

Deployment model: PolicyAware is an adoption-ready open-source framework that teams embed and operate inside their own AI applications, platforms, and CI workflows. It keeps the base install lightweight with native rules-based governance, and uses optional integrations such as Presidio, ProtectAI/Transformers, NeMo Guardrails, Guardrails AI, Haystack, and provider adapters when teams need deeper detection or ecosystem-specific behavior.

Documentation site: https://ktirupati.github.io/policyaware/

Capability docs: docs/capabilities.md Ready-to-use YAML policies: docs/capabilities/ready-to-use-yaml.md Comparison guide: PolicyAware vs guardrails vs AI gateway vs model router Alternatives guide: PolicyAware alternatives for guardrails, AI gateways, model routers, and MCP governance Usage modes: Gateway vs callbacks vs tool governance vs scan Enterprise readiness: enterprise AI governance checklist Limitations: current scope and production validation notes Security model: deny-by-default and layered AI governance Security boundaries: SDK mode vs sidecar/gateway enforcement Examples matrix: choose the right runnable example Compatibility: Python, providers, extras, and integration status Integrations strategy: official vs compatible integrations Benchmarks: lightweight governance benchmarks Policy packs: healthcare, finance, EU AI Act, and SOC 2 starter policies HTTP sidecar: use PolicyAware from Node.js, Go, Java, Rust, and other services Dynamic policy distribution: central HTTP, S3, GCS, and ADLS policy sources with refresh, cache, and emergency revoke patterns Stateful session governance: detect cumulative leakage and repeated tool activity across a session Enterprise hardening: SQLite session state, emergency revokes, checksum pinning, and signed audit traces Policy rollout and trace correlation: shadow policy evaluation, canary enforcement, parent traces, and dashboard Observability templates: Grafana, Prometheus, and OpenTelemetry examples Official GitHub Action: ktirupati/policyaware-action for PolicyAware pull-request scans, annotations, SARIF, and report artifacts Policy contract checks: prevent YAML/tool drift in CI Policy composition: hierarchical global, compliance, tenant, app, and local overrides Runnable contract example: examples/policy-contract-checks Demo outputs: captured terminal output for runnable examples Changelog: release history

Enterprise Core Capabilities

1. Agent Control Plane And MCP Governance

• Action-level tool governance: Evaluates connector names, action names, arguments, user role, tenant, region, and approval requirements before agent tools execute.
• Deny-by-default tool shielding: Supports MCP-style tool policies that block unauthorized or destructive actions unless explicitly allowed.

2. Multi-Engine Security And Guardrail Orchestration

• Unified governance pipeline: Combines data protection, YAML policy enforcement, risk classification, model routing, guardrail adapters, evaluation, and audit logging in one modular Python framework.
• Optional security integrations: Supports optional integrations such as Microsoft Presidio for stronger PII detection, ProtectAI/Transformers-based classifiers for ML signals, NVIDIA NeMo Guardrails, and Guardrails AI adapters.

3. Cost-Aware Runtime Routing And FinOps Controls

• Policy-based model routing: Routes requests across local and external model providers based on task type, risk level, region, provider availability, cost, quality, and policy constraints.
• Token and budget controls: Supports token, budget, and risk-aware limits to help reduce runaway agent loops and uncontrolled model usage.

4. Observability, Audit, And Local Code Scanning

• Audit-ready traces: Records structured policy decisions, risk tiers, reason codes, model choices, evaluation scores, token estimates, and request/response snapshots.
• Observability exporters: Provides live sidecar /metrics, Prometheus-style metrics, OpenTelemetry-shaped events, and audit-trace exports for monitoring and compliance workflows.
• Structured rejection handshakes: Returns canonical blocked-action payloads with decision, reason codes, matched rules, trace IDs, remediation, and telemetry fields so API wrappers do not swallow governance context.
• Dynamic policy retry protection: Applies strict fetch timeouts, refresh TTLs, exponential backoff, jitter, last known-good cache, and emergency fallback policies for central HTTP/S3/GCS/ADLS policy sources.
• PolicyAware Scan CLI: Scans local codebases for PII, PHI, secrets, direct LLM calls, missing tool governance, weak routing controls, audit gaps, and configuration risks, then generates developer-friendly HTML, JSON, SARIF, and Markdown reports.
• Framework callbacks: Includes lightweight LangChain and LlamaIndex callback handlers that aggregate streamed tokens and report policy, risk, leakage, eval, and token-accounting results.
• LangGraph node guard: Adds dependency-free node/state and tool-call governance for graph-based AI agents.
• Haystack governance components: Adds optional Haystack-style components for RAG query governance, output evaluation, and agent tool permission checks.
• Microsoft AGT-style evidence export: Converts PolicyAware policy, tool, gateway, and audit decisions into dependency-free evidence JSON for enterprise agent governance workflows.
• Smart integration recommender: Inspects project signals and user hints to recommend FastAPI, LangChain, LangGraph, Haystack, MCP/tool governance, privacy, guardrails, routing, audit, or scan entry points with reasons.
• Policy packs and sidecar mode: Includes compliance-oriented starter policy packs and a lightweight HTTP sidecar so non-Python services can call PolicyAware.
• Policy/code contract checks: Detects drift between tool-governance YAML and Python tool function signatures before production.

Runtime Flow Summary

flowchart LR
    A["AI App / RAG Pipeline / Agent"] --> B["PolicyAware SDK / CLI / Middleware / Callback"]
    B --> C["Data Protection"]
    C --> D["Risk Classification"]
    D --> E["Policy Decision"]
    E -->|deny| F["Stop"]
    E -->|approval| G["Human Approval"]
    E -->|allow| H["Model Routing"]
    E --> J["MCP Tool Governance"]
    H --> I["Model Provider"]
    J --> K["Tool / Connector"]
    I --> L["Runtime Evaluation"]
    K --> L
    L --> M["Audit Trace / Evidence"]

Read more: Architecture

Which Entry Point Should I Use?

Read more: Usage Modes

Where PolicyAware Fits

Use Guardrails AI or NeMo Guardrails when your main goal is conversational safety, structured outputs, or model response validation.

Use an AI gateway or model router when your main goal is provider abstraction, API key handling, retries, rate limits, fallback, or cost/latency routing.

Use PolicyAware when AI requests and agent actions need governance: user/tenant/context policy, PII/PHI/secrets handling, MCP/tool permissions, approval decisions, model routing after policy approval, evaluations, and audit evidence.

Short version: use PolicyAware when AI actions need governance, not just generation.

Scan Report Preview

Author

Created and maintained by Krishna Kishor Tirupati.

Project links:

• PyPI: policyaware
• GitHub: ktirupati/policyaware
• Documentation: PolicyAware AI Gateway Docs
• LinkedIn: Krishna Tirupati

Feedback And Testimonials

Using PolicyAware in a project, prototype, enterprise AI workflow, security review, or governance evaluation?

Please share feedback, use cases, feature requests, and testimonials through the channels below:

• Private structured feedback form: PolicyAware User Feedback And Testimonials
• Public discussions: GitHub Discussions
• Testimonials and user stories: Show and Tell
• Issues and bugs: GitHub Issues

Helpful feedback includes what you built, which PolicyAware feature you used, what risk or governance gap it helped identify, and what should improve next.

Please do not share secrets, private prompts, PHI, PII, customer data, or confidential internal details.

Contributing And Roadmap

PolicyAware welcomes focused contributions from developers, AI platform engineers, security engineers, and governance practitioners.

• Contributing guide: CONTRIBUTING.md
• Roadmap: ROADMAP.md
• Good first issues: GOOD_FIRST_ISSUES.md
• Security policy: SECURITY.md
• Adoption and impact tracking: ADOPTION.md
• Curated testimonials: TESTIMONIALS.md

Quick Start

pip install policyaware
policyaware about
policyaware feedback
policyaware integrations list
policyaware integrations recommend .
policyaware integrations recommend . --use-case rag --framework haystack --needs "citations pii audit"
policyaware integrations recommend . --html integration-report.html
policyaware doctor
policyaware examples list
policyaware examples run langgraph-agent-governance
policyaware policy packs list
policyaware policy packs copy healthcare-hipaa --out policyaware.yaml
policyaware policy compose-check examples/policy-composition/policy-stack-safe.yaml
policyaware policy compose examples/policy-composition/policy-stack-safe.yaml --out policyaware.composed.yaml
policyaware contract check ./src --policy tool-governance.yaml
policyaware up --policy policyaware.yaml --port 8080
policyaware up --policy-url s3://policy-configs/prod/policyaware.yaml --policy-cache .policyaware/policy-cache.yaml --fallback-policy examples/policies/emergency-fallback-deny.yaml --require-auth
policyaware init
policyaware policy validate policyaware.yaml
policyaware policy migrate policyaware.yaml --to 0.3 --out policyaware.v0.3.yaml
policyaware dev simulate
policyaware risk classify "Email jane@example.com about a patient diagnosis" --domain healthcare
policyaware scan ./mylocalfolder
policyaware scan ./mylocalfolder --json policyaware-scan-report.json --fail-on high
policyaware scan ./mylocalfolder --sarif policyaware.sarif
policyaware scan ./mylocalfolder --markdown policyaware-scan-report.md
policyaware scan ./mylocalfolder --baseline policyaware-baseline.json
policyaware scan ./mylocalfolder --config examples/policyaware-scan.yaml
policyaware scan ./mylocalfolder --ruleset ai-agent-security
policyaware scan ./mylocalfolder --diff --diff-base origin/main
policyaware scan ./mylocalfolder --format html,json,sarif,markdown
policyaware guards list examples/full-stack-guardrails/policy.yaml

For GitHub pull-request checks, use the official action:

- uses: actions/checkout@v4
- uses: ktirupati/policyaware-action@v1

Advanced CI/CD flows can combine the action with policyaware policy validate, policyaware policy compose-check, policyaware policy compose, policyaware contract check, and SARIF-producing policyaware scan commands.

Installation Profiles

The default install is intentionally lightweight. It includes the core CLI, local scanner, policy engine, routing abstractions, cost/risk governance primitives, audit/eval contracts, and YAML policy support.

pip install policyaware

Install optional integrations only when you need them:

pip install "policyaware[privacy]"     # Presidio + spaCy privacy detection
pip install "policyaware[guardrails]"  # NeMo Guardrails + Guardrails AI
pip install "policyaware[haystack]"    # Haystack RAG/agent integration environment
pip install "policyaware[providers]"   # Provider extras such as Bedrock boto3
pip install "policyaware[ml]"          # Transformers/Torch classifiers
pip install "policyaware[onnx]"        # ONNX runtime path for supported classifiers
pip install "policyaware[all]"         # All optional integrations

Backward-compatible aliases are also available:

pip install "policyaware[presidio]"
pip install "policyaware[nemo]"
pip install "policyaware[guardrails-ai]"
pip install "policyaware[full]"

For local development from this repository:

pip install -e ".[dev]"
policyaware policy test examples/policies/basic.yaml
policyaware policy validate examples/policies/basic.yaml
policyaware risk classify "Summarize this patient diagnosis" --domain healthcare
policyaware tools check examples/policies/tool-governance.yaml --agent code_assistant --connector github --action create_pr
policyaware eval run examples/evals/support_rag.yaml
policyaware scan . --out policyaware-scan-report.html
policyaware scan . --include ".py,.yaml,.json" --exclude "tests,fixtures"
policyaware scan . --write-baseline policyaware-baseline.json
policyaware scan . --config examples/policyaware-scan.yaml --format html,json,sarif,markdown

For copy-pasteable end-to-end examples, see Working Examples.

Local code scan docs: policyaware scan

Official GitHub integration: ktirupati/policyaware-action

GitHub Action docs: PolicyAware policy CI/CD with GitHub Actions

CLI usability docs: doctor, examples, migration, recommendation reports

Microsoft AGT-style interop example: examples/microsoft-agt-interop

LangGraph governance example: examples/langgraph-agent-governance

Enterprise control-plane demo: examples/enterprise-ai-control-plane

Generate A Starter Policy

Create a NIST-aligned baseline starter policy in the current directory:

policyaware init
policyaware policy validate policyaware.yaml

Use a custom path or overwrite intentionally:

policyaware init --out config/policyaware.yaml
policyaware init --out policyaware.yaml --force

The generated template is deny-by-default and includes baseline rules for PII/PHI/secrets handling, risky MCP/tool command blocking, approval for side-effecting tool actions, token budget caps, and high-iteration agent workflows.

LangChain And LlamaIndex Callbacks

Use callbacks when you already have an LLM framework pipeline and want PolicyAware governance results without changing the model call.

from policyaware.integrations.langchain import PolicyAwareCallbackHandler

policyaware_callback = PolicyAwareCallbackHandler(config="policyaware.yaml")

response = chain.invoke(
    {"question": "Summarize this customer ticket."},
    config={"callbacks": [policyaware_callback]},
)

result = policyaware_callback.last_result
print(result.policy_decision.decision)
print(result.risk.tier)
print(result.output_findings.contains_sensitive)

Streaming-friendly manual example:

from policyaware.integrations.langchain import PolicyAwareCallbackHandler

handler = PolicyAwareCallbackHandler(config="policyaware.yaml")
handler.on_llm_start(prompts=["Email jane@example.com with the ticket summary."])

for token in ["Safe ", "summary ", "without ", "private ", "data."]:
    handler.on_llm_new_token(token)

result = handler.on_llm_end()
print(result.to_dict())

LlamaIndex-style callbacks are also available:

from policyaware.integrations.llamaindex import PolicyAwareCallbackHandler

handler = PolicyAwareCallbackHandler(config="policyaware.yaml")
handler.on_event_start(payload={"query_str": "Answer with citations from policy documents."})
handler.on_llm_new_token("The policy requires citation review [doc-1].")
result = handler.on_event_end(payload={})

More details: LangChain and LlamaIndex callback integrations

Copy-Paste Examples

• FastAPI LLM policy middleware: protect a FastAPI /chat endpoint with policy checks before model execution.
• LangChain policy guardrails: wrap a chain-style LLM call with deny-by-default policy, PII redaction, and secret blocking.
• MCP tool permission gateway: govern connector-level and action-level tool permissions for agent workflows.
• Haystack RAG governance: add PolicyAware query, output, and tool governance around Haystack-style RAG and agent workflows.
• PII redaction policy: inspect and redact sensitive text before model execution.
• Regulated RAG assistant: require citations and stricter controls for healthcare-style RAG.
• Provider routing by risk: route public-safe requests to low-cost models and high-risk requests to approved models.
• Audit trace viewer: write audit traces and generate a local HTML trace viewer.
• Approval workflow hooks: send high-risk requests to approval instead of calling a model.
• Local code scan: scan local AI app code and generate an HTML governance report.
• Full-stack guardrails: orchestrate NeMo Guardrails, Guardrails AI, or custom validators as input/output guards.

Captured terminal output for the runnable examples is available in docs/demo-outputs.md.

Articles

• PolicyAware vs Guardrails vs AI Gateways vs Model Routers
• Build a Policy-Aware AI Gateway in Python
• Stop Shipping AI Features Without Guardrails

from policyaware import Gateway, GatewayRequest

gateway = Gateway.from_policy_file("examples/policies/basic.yaml")

response = gateway.chat(
    GatewayRequest(
        tenant="acme",
        app="claims-assistant",
        user={"id": "u_123", "role": "claims_adjuster"},
        context={"region": "us", "task_type": "summarization", "risk": "low"},
        messages=[{"role": "user", "content": "Summarize claim ACME-42."}],
    )
)

print(response.content)
print(response.policy.decision)
print(response.policy.reason_codes)
print(response.trace_id)

Architecture

Application / Agent / RAG App
        |
        v
PolicyAware SDK / Middleware
        |
        v
Identity + Context Resolver
        |
        v
Policy Decision Engine -> Data Protection Engine -> Model Router -> Provider/Tool
        |
        v
Runtime Evaluation -> Audit Trace -> Response

Repository Layout

src/policyaware/
  audit.py              Request traces and audit export records
  cli.py                policyaware CLI
  data_protection.py    PII/PHI/secret detection and redaction
  evals.py              Offline and runtime evaluation primitives
  gateway.py            Main SDK facade
  models.py             Core typed contracts
  policy.py             Deny-by-default policy engine
  providers.py          Provider abstraction and local simulated provider
  routing.py            Policy-aware model routing
  integrations/         FastAPI, Flask, LangChain, LlamaIndex shims
examples/
  policies/
  evals/
tests/

Policy Example

id: basic_enterprise_policy
default: deny

rules:
  - name: allow_low_risk_support
    effect: allow
    when:
      user.role_in: ["support_agent", "claims_adjuster"]
      request.risk_in: ["low", "medium"]
      data.contains_secrets: false

  - name: redact_pii_for_non_privileged_users
    effect: transform
    action: redact
    when:
      data.contains_pii: true
      user.role_not_in: ["privacy_admin", "compliance_officer"]

  - name: require_approval_for_high_risk
    effect: require_approval
    when:
      request.risk: "high"

Development Status

This is a production-oriented open-source framework: the core extension points and executable behavior are present, while teams can connect their preferred identity systems, workflow tools, storage backends, dashboards, and enterprise review processes around it.

v0.2 MVP Capabilities

• Deterministic risk classification: low, medium, high, critical.
• Explainable policy decisions with reason codes and remediation.
• Replayable audit trace snapshots.
• Audit bundle generation.
• Tool governance policies for MCP-style connectors and actions.
• Governance-aware eval report schema.
• Provider adapters for OpenAI-compatible APIs, Azure OpenAI, Anthropic, Bedrock, Vertex AI, Ollama, and vLLM.
• Optional ML signal integrations for Presidio PII detection, ProtectAI prompt-injection detection, and custom Transformers domain/risk classifiers.
• Optional NeMo Guardrails and Guardrails AI adapters for full-stack guardrail orchestration.
• Fast local code scanner and HTML recommendation report.
• SQLite audit storage and static trace viewer.
• Prometheus text and OpenTelemetry-shaped JSON exporters.
• File and webhook approval hooks.
• Executable golden dataset policy checks.

Third-Party ML Models

Optional ML integrations may download third-party models at runtime. PolicyAware does not bundle model weights. Review and accept the license or access terms for any model you configure, especially gated Hugging Face models.

Recommended GitHub Topics

For discovery, use repository topics such as llm, ai-gateway, llm-governance, guardrails, rag, mcp, ai-agents, pii-redaction, model-routing, audit, python, and open-source.

License

Apache-2.0