RAGOne

A lightweight, ModernBERT-based model for evaluating retrieved context in RAG pipelines.

RAGOne evaluates retrieved context across four tasks: answerability, evidence quality, contradiction detection, and relevance classification.

• Hugging Face model: Hariharan05/ragone
• PyPI package: ragone-ai
• Python import: from ragone import RAGOne

Installation

pip install ragone-ai

Quick start

from ragone import RAGOne

model = RAGOne.from_pretrained("Hariharan05/ragone")

result = model.predict(
    query="What is the refund period?",
    context="Customers can request a refund within 30 days.",
    task="answerability",
    options=["yes", "no"],
)

print(result)

On first use, the model files are downloaded from Hugging Face. The package includes the inference code; model weights are hosted separately in the Hugging Face repository.

Supported tasks

Use labels consistent with the model's training conventions.

Example output

{
    "prediction": "yes",
    "scores": [
        {"label": "yes", "score": 0.89},
        {"label": "no", "score": 0.11}
    ],
    "latency_ms": 85.0,
    "task": "answerability"
}

Scores and latency above are illustrative, not guaranteed results. Scores are normalized across candidate options and are not necessarily calibrated probabilities of correctness.

Model and training

RAGOne uses a fine-tuned answerdotai/ModernBERT-base encoder and a neural scoring head to rank candidate labels.

Reported training details:

• Dataset: 5,074 records
• Training: 5 epochs
• Best reported validation loss: 0.2697
• Reported final validation accuracy: 94.591%
• Separate evaluation: 37/40 correct (92.5%)

The separate evaluation used 10 examples per task. These results are preliminary and do not establish performance on all real-world RAG systems. The training and evaluation metrics come from different evaluations and should not be compared as if they were the same test.

Intended use

RAGOne is intended for research, experimentation, and development of RAG evaluation workflows. It can help assess retrieved-context relevance, answerability, evidence quality, and contradiction.

It does not perform document retrieval or answer generation and should not be treated as a definitive source of truth.

Limitations

• Performance may vary by domain, language, and retrieval strategy.
• Inputs are limited to the model's configured token length.
• Results depend on the candidate labels supplied.
• Softmax scores should not be interpreted as calibrated confidence without additional validation.
• Evaluate on independent, representative datasets before production use.

License

RAGOne is licensed under the Apache License 2.0.

You may use, modify, and distribute this software under its terms. See the LICENSE file or https://www.apache.org/licenses/LICENSE-2.0 for the full license text.