trustrag-eval

RAGAS-based evaluation pipeline with trust-specific metrics for TrustRAG.

Getting Started

# Install in development mode
pip install -e packages/trustrag-eval

# Run tests
cd packages/trustrag-eval && pytest tests/ -v

Metrics Explained

RAGAS Metrics (industry-standard)

Trust-Specific Metrics (TrustRAG)

Usage

from trustrag_eval import load_synthetic_queries, compute_trust_metrics

# Load benchmark dataset
queries = load_synthetic_queries("eval/synthetic_queries.json")

# Compute trust distribution from results
results = [{"trust_score": 85}, {"trust_score": 72}, ...]
dist = compute_trust_metrics(results)
print(f"Median trust: {dist['p50']}, Flagged: {dist['flagged_pct']:.0%}")