rag_eval_tool
rag_eval_tool is a Python library designed for comprehensive evaluation of Retrieval-Augmented Generation (RAG) systems and Large Language Models (LLMs). It offers a wide array of metrics to evaluate generated text across fluency, diversity, semantic similarity, readability, hallucination, and bias detection.
Features
- Text Similarity Metrics: BLEU, ROUGE, BERTScore, METEOR, CHRF.
- Fluency and Coherence: Perplexity using GPT-2.
- Lexical Metrics: Diversity and Entropy.
- Readability Scores: Flesch Reading Ease, Flesch-Kincaid Grade Level.
- Bias Detection: Racial bias evaluation using a zero-shot classification model.
- Hallucination Metrics: Quantify unsupported content in generated text.
- Precision/Recall/F1: Measure token overlap between response and reference.
Installation
Install the library directly from PyPI:
pip install rag_eval_tool
1 maintainer
Metadata
1 maintainer
Release files for rag-eval-tool 0.1.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| rag_eval_tool-0.1.9.tar.gz | 4.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| rag_eval_tool-0.1.9-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 8.8 kB
Release files / rag_eval_tool-0.1.9.tar.gz
| Download URL | rag_eval_tool-0.1.9.tar.gz |
|---|---|
| Size | 4.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
92663fcce28c4f7e5f4206d4e533a9133d18f71cc5b79709a0f924d64d2ffe59
|
|
BLAKE2b-256 checksum How to use checksums |
62200958f38ca47dfd510e97d7e199cf1ddfa0c1d1ee2d1254e569fb7080c194
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.12.7
|
Release files / rag_eval_tool-0.1.9-py3-none-any.whl
| Download URL | rag_eval_tool-0.1.9-py3-none-any.whl |
|---|---|
| Size | 4.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f7629b9c209fcaeefb92eba5e12f16704ff7c775149f919cfa5f763f4aa4671f
|
|
BLAKE2b-256 checksum How to use checksums |
bcfd11ee327b7cc6d0d99560817479b45e522ea8afca657585e7bc06fb8c03fc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.0.1 CPython/3.12.7
|
