ragrefine

Evidence-first post-retrieval refinement for RAG systems.

ragrefine improves an existing retriever's candidate set before it reaches an LLM. It reranks, fuses, deduplicates, and selects context while preserving the original evidence and recording an auditable trace.

retriever -> Top-N candidates -> ragrefine -> refined Top-K + trace -> LLM

The core library is framework- and retriever-agnostic. Haystack and LangChain support are available as optional integrations.

Core deliverables

Output candidates always originate from the supplied input pool; ragrefine does not invent or rewrite evidence.

Quick start

Python 3.12+ is required.

pip install ragrefine

Or, from this checkout:

uv sync

from ragrefine import Candidate, CandidateSet, Refiner

candidates = CandidateSet(
    name="my-retriever",
    candidates=(
        Candidate(
            id="doc-1",
            text="Python 3.12 improves f-string parsing.",
            retrieval_rank=1,
            retrieval_score=0.91,
        ),
        Candidate(
            id="doc-2",
            text="Python 3.11 introduced exception groups.",
            retrieval_rank=2,
            retrieval_score=0.84,
        ),
    ),
)

result = Refiner().refine(
    query="Which Python version improves f-string parsing?",
    candidate_set=candidates,
    top_k=1,
)

print(result.candidates[0].candidate.id)
print(result.trace)

Refiner() preserves the original ranking. Explicit RefinerConfig settings enable lexical, pattern, neural, or fused profiles. See the technical design for the configuration contract.

Haystack integration

uv sync --extra haystack

The optional integration provides document conversion, a serializable RagRefineComponent, registered refiner profiles, prompt construction, and a complete Haystack pipeline builder. The RAG pipeline guide contains the runnable recipe and customization points.

LangChain integration

pip install "ragrefine[langchain]"

This requires LangChain 1.x (langchain-core>=1.6.9). With the repository's current pins it cannot be installed in the same environment as the ragas extra (the ragas extra pins langchain-openai<1.0).

from ragrefine import Refiner, RefinerConfig
from ragrefine.config import ChannelConfig
from ragrefine.integrations.langchain import RagRefineCompressor
from ragrefine.ranking.lexical import LexicalRanker

refiner = Refiner(
    lexical_ranker=LexicalRanker(),
    config=RefinerConfig(
        original=ChannelConfig(enabled=False), lexical=ChannelConfig(enabled=True)
    ),
)
compressor = RagRefineCompressor(refiner=refiner, top_k=5)
# documents only
docs = compressor.compress_documents(retrieved_docs, "my query")

# documents + RefinementTrace
docs, trace = compressor.refine("my query", retrieved_docs)

• Returned documents are the original objects. They are never copied or edited, and the output is always a subset of the input.
• Documents without an id get a positional id lc:<n>. Duplicate ids raise ValueError.
• Original retrieval scores are not read from LangChain metadata, so they are not carried into the trace.
• To plug into a retriever, use ContextualCompressionRetriever(base_compressor=compressor, base_retriever=retriever) (that class now ships in the langchain-classic package), or wrap the compressor in a runnable: RunnableLambda(lambda x: compressor.compress_documents(x["documents"], x["query"])).

This integration is plumbing only; it makes no retrieval-quality claim. See the evaluation section for what has been measured.

Evaluation summary

Post-retrieval comparisons reuse the same frozen first-stage Top-50 candidates. Retrieval is not regenerated between profiles, and candidate-pool Recall@50 is a fixed ceiling rather than a refinement gain.

The end-to-end Ragas A/B used 13 paired DeepSeek queries. It is statistically inconclusive: the positive directional score differences have 95% paired bootstrap confidence intervals spanning zero, and both context metrics are saturated at 1.0. The run establishes neither an end-to-end benefit nor a regression. See the retained A/B report.

Machine-readable artifacts are the source of truth. No experiment is described as improved unless its recorded measurements support that claim.

Documentation

Detailed material removed from this README remains in the repository's focused documents:

• Product requirements — supported scope and product boundary
• Technical hypotheses — testable claims and current evidence
• Technical design — architecture, configuration, selection, and tracing contracts
• Evaluation contract — frozen-pool rules, metrics, profiles, and reporting policy
• Benchmark guide — execution and reproduction commands
• Retained artifact manifest — checksummed benchmark inputs and verification
• Haystack pipeline guide — integration and serialization
• Ragas evaluation guide — paired A/B design and analysis
• v0.2 release record — delivered scope, validation, and limitations

Scope

ragrefine owns post-retrieval candidate refinement, context selection, and tracing. It deliberately does not own ingestion, parsing, chunking, embeddings, vector databases, first-stage retrieval, LLM generation, agents, or deployment.

Development

uv sync --extra haystack --extra ragas
uv run ruff check .
uv run ruff format --check .
uv run mypy src/
uv run pytest
uv build

Heavy model and benchmark dependencies remain optional so the core package has no runtime dependencies.

License

MIT