RAG Evaluator
Overview
RAG Evaluator is a Python library for evaluating Retrieval-Augmented Generation (RAG) systems. It provides various metrics to evaluate the quality of generated text against reference text.
Installation
You can install the library using pip:
pip install rag-evaluate
Usage: Here's how to use the RAG Evaluator library:
from rag_evaluate import RAG_Evaluator
Initialize the evaluator
evaluator = RAG_Evaluator()
Input data
question = "What are the causes of climate change?." generated_text = "Climate change is caused by human activities." context = "Human activities such as burning fossil fuels cause climate change."
Evaluate the response
bleu_score = evaluator.bleu_score(question, generated_text, context)
rouge_score = evaluator.rouge_score(question, generated_text, context)
bert_score = evaluator.bert_score(question, generated_text, context)
Print the results
print(bleu_score)
print(rouge_score)
print(bert_score)
The RAG Evaluator provides the following metrics:
BLEU (0-100): Measures the overlap between the generated output and reference text based on n-grams.
0-20: Low similarity, 20-40: Medium-low, 40-60: Medium, 60-80: High, 80-100: Very high.
ROUGE-1 (0-1): Measures the overlap of unigrams between the generated output and reference text.
0.0-0.2: Poor overlap, 0.2-0.4: Fair, 0.4-0.6: Good, 0.6-0.8: Very good, 0.8-1.0: Excellent.
BERT Score (0-1): Evaluates the semantic similarity using BERT embeddings (Precision, Recall, F1).
0.0-0.5: Low similarity, 0.5-0.7: Moderate, 0.7-0.8: Good, 0.8-0.9: High, 0.9-1.0: Very high.
Contributer
Biplab Sil.
Metadata
Release files for rag-evaluate 0.5.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| rag_evaluate-0.5.0.tar.gz | 2.5 kB | Without provenance | Details for rag_evaluate-0.5.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| rag_evaluate-0.5.0-py3-none-any.whl | Python 3 | none | any | Details for rag_evaluate-0.5.0-py3-none-any.whl |
Total release size: 5.2 kB
Release files / rag_evaluate-0.5.0.tar.gz
| Download URL | rag_evaluate-0.5.0.tar.gz |
|---|---|
| Size | 2.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
360c3683ef4b303aca26885538ff2b41759b574d40bed1bab7dfddbcd6842949
|
|
BLAKE2b-256 checksum How to use checksums |
dd7be0e26464162454442d06ddd7b97cacc0586e898045c05e1f3047be5ae6a0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.2
|
Release files / rag_evaluate-0.5.0-py3-none-any.whl
| Download URL | rag_evaluate-0.5.0-py3-none-any.whl |
|---|---|
| Size | 2.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f3ccca3867ed727fc98440525783978ac2ce701deb65a5f761cbce618ea507e2
|
|
BLAKE2b-256 checksum How to use checksums |
5946416e08b3cd2cc605e49d253fca33f161d4e75d0dddf09424995306cafae3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.2
|
Security for rag-evaluate 0.5.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.