rag-eval-kit-avan

A lightweight Python wrapper for evaluating RAG (Retrieval-Augmented Generation) pipelines using Ragas.

If you've built a RAG system with LangChain, LlamaIndex, or plain Python, you know that "it works" is not enough. You need to measure Faithfulness, Answer Relevancy, Context Precision, and Context Recall. This package wraps those Ragas metrics into a single function call.

Installation

pip install rag-eval-kit-avan

Usage

from rag_eval_kit import evaluate_rag

questions = ["What is RAG?", "What is a vector database?"]
answers = [
    "RAG stands for Retrieval-Augmented Generation.",
    "A vector database stores embeddings for similarity search.",
]
contexts = [
    ["RAG is a technique that combines retrieval with generation..."],
    ["Vector databases like FAISS and ChromaDB store high-dimensional vectors..."],
]
ground_truths = [
    "RAG stands for Retrieval-Augmented Generation.",
    "A vector database stores embeddings and supports similarity search.",
]

results = evaluate_rag(questions, answers, contexts, ground_truths)
print(results)

Why This Exists

Most RAG tutorials stop at "here's how to build a chain." Nobody tells you how to prove your RAG pipeline is actually working. This package makes it a one-liner.

Built as part of an Eval-First RAG System project.

License

MIT