ragrefine

Evidence-first post-retrieval refinement for RAG systems.

ragrefine improves an existing retriever's candidate set before it reaches an LLM. It reranks, fuses, deduplicates, and selects context while preserving the original evidence and recording an auditable trace.

retriever -> Top-N candidates -> ragrefine -> refined Top-K + trace -> LLM

The core library is framework- and retriever-agnostic. Haystack support is available as an optional integration.

Core deliverables

Output candidates always originate from the supplied input pool; ragrefine does not invent or rewrite evidence.

Quick start

Python 3.12+ is required. From this checkout:

uv sync

from ragrefine import Candidate, CandidateSet, Refiner

candidates = CandidateSet(
    name="my-retriever",
    candidates=(
        Candidate(
            id="doc-1",
            text="Python 3.12 improves f-string parsing.",
            retrieval_rank=1,
            retrieval_score=0.91,
        ),
        Candidate(
            id="doc-2",
            text="Python 3.11 introduced exception groups.",
            retrieval_rank=2,
            retrieval_score=0.84,
        ),
    ),
)

result = Refiner().refine(
    query="Which Python version improves f-string parsing?",
    candidate_set=candidates,
    top_k=1,
)

print(result.candidates[0].candidate.id)
print(result.trace)

Refiner() preserves the original ranking. Explicit RefinerConfig settings enable lexical, pattern, neural, or fused profiles. See the technical design for the configuration contract.

Haystack integration

uv sync --extra haystack

The optional integration provides document conversion, a serializable RagRefineComponent, registered refiner profiles, prompt construction, and a complete Haystack pipeline builder. The RAG pipeline guide contains the runnable recipe and customization points.

Evaluation summary

Post-retrieval comparisons reuse the same frozen first-stage Top-50 candidates. Retrieval is not regenerated between profiles, and candidate-pool Recall@50 is a fixed ceiling rather than a refinement gain.

The end-to-end Ragas A/B used 13 paired DeepSeek queries. It is statistically inconclusive: the positive directional score differences have 95% paired bootstrap confidence intervals spanning zero, and both context metrics are saturated at 1.0. The run establishes neither an end-to-end benefit nor a regression. See the retained A/B report.

Machine-readable artifacts are the source of truth. No experiment is described as improved unless its recorded measurements support that claim.

Documentation

Detailed material removed from this README remains in the repository's focused documents:

• Product requirements — supported scope and product boundary
• Technical hypotheses — testable claims and current evidence
• Technical design — architecture, configuration, selection, and tracing contracts
• Evaluation contract — frozen-pool rules, metrics, profiles, and reporting policy
• Benchmark guide — execution and reproduction commands
• Retained artifact manifest — checksummed benchmark inputs and verification
• Haystack pipeline guide — integration and serialization
• Ragas evaluation guide — paired A/B design and analysis
• v0.2 release record — delivered scope, validation, and limitations

Scope

ragrefine owns post-retrieval candidate refinement, context selection, and tracing. It deliberately does not own ingestion, parsing, chunking, embeddings, vector databases, first-stage retrieval, LLM generation, agents, or deployment.

Development

uv sync --extra haystack --extra ragas
uv run ruff check .
uv run ruff format --check .
uv run mypy src/
uv run pytest
uv build

Heavy model and benchmark dependencies remain optional so the core package has no runtime dependencies.

License

MIT