hardrag-core 0.1.0

Evaluation-First Control Layer for Enterprise RAG Systems

HardRAG Core

Evaluation-First Control Layer for Enterprise RAG Systems

HardRAG provides production-ready guardrails and evaluation metrics for RAG (Retrieval-Augmented Generation) systems, with a focus on enterprise governance, audit-readiness, and policy compliance.

Why HardRAG?

Most RAG systems work. Very few can be trusted.

HardRAG solves this by adding:

• ✅ Guardrails - PII detection, toxicity filtering, grounding validation
• ✅ Evaluation - Faithfulness, relevance, policy compliance metrics
• ✅ Audit Trails - Complete request/response logging for compliance
• ✅ DSPy-Powered - Self-improving, optimizable guardrails

Quick Start

pip install hardrag-core

from hardrag import HardRAGGuard

# Initialize guard with guardrails
guard = HardRAGGuard(
    guardrails=["pii", "grounding", "toxicity"],
    evaluation_mode="strict"
)

# Validate RAG output
result = guard.validate(
    query="What is ACME Corp's revenue?",
    retrieved_sources=[
        "ACME Corp reported $10M revenue in Q4 2023."
    ],
    llm_output="ACME Corp's revenue was $10M in Q4 2023."
)

print(result.is_valid)  # True
print(result.violations)  # []
print(result.audit_trail)  # Full evidence for audit

Features

Guardrails

• PII Detection - Prevent leakage of sensitive data (emails, SSN, credit cards, phone numbers)
  • Microsoft Presidio integration
  • 30+ PII types detected
  • Anonymization support
  • Configurable allow-lists
• Grounding Check - Verify outputs are supported by retrieved sources
  • DSPy-powered intelligent verification
  • Claim-level validation
  • Source attribution tracking
• Toxicity Filter - Block harmful or inappropriate content
  • ML-based detection (transformers)
  • Rule-based fallback
  • Severity levels (LOW/MEDIUM/HIGH/CRITICAL)
  • 6 toxicity categories
• Policy Compliance - Enforce custom business rules (coming soon)

Evaluation Metrics

• Faithfulness - How well is the answer grounded in sources?
• Relevance - Does the answer address the query?
• Attribution - Are sources properly cited?

Enterprise Features

• Audit Trails - Complete logging for compliance
• Custom Guardrails - Define your own validation logic
• Batch Evaluation - Process multiple outputs efficiently
• Integration - Works with LangChain, LlamaIndex, or custom RAG

Installation

pip install hardrag-core

From Source

git clone https://github.com/yourusername/hardrag-core
cd hardrag-core
pip install -e .

Usage

Basic Guardrails

PII Detection

from hardrag.guardrails import PIIGuardrail

from hardrag import HardRAGGuard

guard = HardRAGGuard(guardrails=["pii"])
result = guard.validate(
    query="What's John's email?",
    retrieved_sources=["John's email is [e-mail]"],
    llm_output="John's email is [e-mail]"
)

if not result.is_valid:
    print(f"Violations: {result.violations}")
    print(f"Safe alternative: {result.anonymized_text}")

Toxicity Detection

from hardrag.guardrails import ToxicityGuardrail

toxicity_guard = ToxicityGuardrail(threshold=0.7)
result = toxicity_guard.validate(
    text="Your message here",
    block_if_toxic=True
)

if not result["is_valid"]:
    print(f"Toxic content detected: {result['categories']}")

Grounding Verification

from hardrag.guardrails import GroundingGuardrail
import dspy

# Configure DSPy (required)
dspy.settings.configure(lm=dspy.OpenAI(model="gpt-3.5-turbo"))

grounding_guard = GroundingGuardrail(threshold=0.8)
result = grounding_guard(
    retrieved_chunks=["Source text..."],
    llm_output="Generated answer..."
)

print(f"Grounded: {result['is_grounded']}")
print(f"Score: {result['score']}")

With LangChain

from langchain.chains import RetrievalQA
from hardrag.integrations.langchain import HardRAGCallback

chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=retriever,
    callbacks=[HardRAGCallback(guardrails=["pii", "grounding"])]
)

Evaluation

from hardrag.evaluation import FaithfulnessEvaluator

evaluator = FaithfulnessEvaluator()
score = evaluator.evaluate(
    sources=["Source text..."],
    output="Generated answer..."
)

print(f"Faithfulness score: {score}")  # 0.0 - 1.0

Architecture

HardRAG Core
├── Guardrails Layer (Pre-validation)
│   ├── PII Detection
│   ├── Toxicity Filtering
│   └── Grounding Validation
├── Evaluation Layer (Post-validation)
│   ├── Faithfulness Metrics
│   ├── Relevance Scoring
│   └── Attribution Check
└── Audit Layer (Compliance)
    ├── Request/Response Logging
    └── Evidence Generation

Roadmap

• Core guardrails (PII, toxicity, grounding)
• Policy compliance engine
• Advanced evaluation metrics
• REST API
• Dashboard UI
• Multi-language support

Contributing

Contributions welcome! See CONTRIBUTING.md

License

MIT License - see LICENSE

Links

• Documentation: docs.hardrag.dev
• Website: hardrag.com
• Discord: Join community

---

Built with ❤️ for Enterprise AI Governance