validate-llm 0.1.2

A composable validation framework for LLM inputs and outputs

A plug and play Python library for developers to wrap around any LLM to safeguard the inputs and outputs. It detects accuracy, relevancy, toxicity, privacy, and bias checks in one pipeline.

Documentation →

---

Install

pip install validate-llm

Requires Python 3.11+. Optional extras:

pip install "validate-llm[demo]"   # FastAPI demo server + web UI
pip install "validate-llm[test]"   # pytest + datasets
pip install "validate-llm[dev]"    # everything

Quick start

from llm_validation_framework import ValidationFramework, LLMProvider, Pipe
from llm_validation_framework import ToxicityAgent, PrivacyAgent, AccuracyAgent
from llm_validation_framework.config_loader import load_api_key

llm = LLMProvider(provider="anthropic", model="claude-haiku-4-5-20251001", key=load_api_key())

vf = ValidationFramework(
    llm=llm,
    input_guardrail=Pipe(steps=[ToxicityAgent()], verbose=False),
    output_guardrail=Pipe(steps=[ToxicityAgent(), PrivacyAgent(), AccuracyAgent()], verbose=False),
)

result = vf.validate("What is the Pacific Ocean?")
print(result["status"], result["score"])  # PASS 0.87

validate() returns a structured dict with status, score, and per-agent results for both the input and output guardrails. See the docs for the full schema.

Demo

The demo is a FastAPI backend + static web UI.

# Terminal 1
uvicorn demo.api_server:app --host 127.0.0.1 --port 5050

# Terminal 2
python demo/serve_ui.py

Open http://127.0.0.1:8000.

Agents

Agent	What it does	Needs API key
ToxicityAgent	Three-layer check: profanity filter → toxicity model → semantic similarity	No
PrivacyAgent	Regex scan for SSN, credit cards, API keys; optional system prompt leakage detection	No
AccuracyAgent	LLM-as-a-judge factual accuracy + relevancy, with optional RAG grounding	Yes
RelevancyAgent	LLM-as-a-judge check that the answer addresses the question	Yes
BiasAgent	LLM-as-a-judge scan for stereotypes and discriminatory language	Yes

ToxicityAgent and PrivacyAgent run fully locally with no external calls.

Config

export ANTHROPIC_API_KEY=your-key

Or create a config.ini at the repo root (gitignored):

[ANTHROPIC]
API_KEY=your-key

Supported providers follow litellm's naming.

RAG grounding

Pass a retriever to AccuracyAgent to ground factual checks against your own corpus:

from llm_validation_framework import AccuracyAgent, RAGProvider

accuracy = AccuracyAgent(rag=RAGProvider(your_vectorstore.as_retriever()))

See the RAG Integration guide for a full walkthrough.

Contributors

• Hitha Shri Nagaruru
• James Wu
• Lewis Lui
• Thomas Yeoh

License

MIT — see LICENSE