ragcompress

Extractive, provenance-preserving context compression for RAG pipelines.

ragcompress is a drop-in reranker/compression stage between your retriever and your generator LLM. Given a query and a list of retrieved chunks, it selects the minimal set of original, verbatim sentences needed to answer the query — instead of summarizing (which can hallucinate) or truncating to a fixed top-k (which wastes budget on irrelevant sentences inside relevant chunks).

retriever ──> [ragcompress] ──> generator LLM
              keeps only the sentences the query needs,
              verbatim, with per-sentence source attribution

Why

• Extractive, never abstractive. Every output sentence exists verbatim in an input chunk. Nothing is paraphrased, so the compression stage cannot hallucinate — and citations stay valid.
• Provenance built in. Each kept sentence carries its source doc_id and chunk_id, queryable on the result object and optionally inlined as [doc_id] markers in the context string.
• Adaptive token budget. Instead of a fixed top-k, an elbow-detection budget keeps sentences until relevance falls off a cliff — short context for easy questions, longer for multi-hop ones.
• Framework-native adapters. A LlamaIndex NodePostprocessor and a LangChain BaseDocumentCompressor (drop-in for ContextualCompressionRetriever).
• CPU-friendly. The default scorer is a small pretrained cross-encoder (ms-marco-MiniLM-L-6-v2) plus a MiniLM embedder for redundancy removal — no fine-tuning, no GPU required.

Install

pip install ragcompress                # core
pip install ragcompress[llamaindex]    # + LlamaIndex adapter
pip install ragcompress[langchain]     # + LangChain adapter
pip install ragcompress[attention]     # + attention-based scorer (local HF models)
pip install ragcompress[benchmarks]    # + benchmark suite deps

Quickstart

from ragcompress import Chunk, Compressor, FixedBudget

chunks = [
    Chunk(doc_id="dune-wiki", chunk_id="c0", text="Dune is a novel by Frank Herbert. ..."),
    # ... more retrieved chunks
]

compressor = Compressor(budget=FixedBudget(max_tokens=500))
result = compressor.compress("Who wrote Dune?", chunks)

result.text  # compressed context, ready for your prompt
result.compression_ratio  # e.g. 0.18
result.sentences[0].source_doc_id  # provenance for every kept sentence
result.provenance()  # [(doc_id, chunk_id), ...] in output order

Swap in the adaptive budget:

from ragcompress import AdaptiveBudget, Compressor

compressor = Compressor(budget=AdaptiveBudget(min_tokens=64, max_tokens=1024))

How it works

1. Split — chunks are segmented into sentences with provenance tags. Code fences, markdown tables, and list items are kept atomic (never split mid-fence).
2. Score — each (query, sentence) pair is scored by a cross-encoder reranker; an MMR-style redundancy pass (cosine similarity between candidates) penalizes near-duplicates of higher-ranked sentences so diverse evidence wins.
3. Budget — a controller selects sentences: FixedBudget (top-k and/or token cap) or AdaptiveBudget (elbow detection on the score distribution, bounded by a min_tokens/max_tokens range).
4. Reassemble — survivors are deduplicated, restored to original reading order (doc → chunk → sentence position, which reads more coherently to the generator than score order), and joined into the final context with optional [doc_id] provenance markers.

Scoring backends

from ragcompress.scoring import AttentionScorer

compressor = Compressor(scorer=AttentionScorer(model="path/or/hf-id-of-local-model"))

Framework adapters

LlamaIndex (pip install ragcompress[llamaindex]):

from ragcompress.adapters.llamaindex import RagCompressPostprocessor

postprocessor = RagCompressPostprocessor.from_compressor(compressor)  # mode="merge" | "filter"
query_engine = index.as_query_engine(node_postprocessors=[postprocessor])

LangChain (pip install ragcompress[langchain]):

from langchain.retrievers import ContextualCompressionRetriever
from ragcompress.adapters.langchain import RagCompressDocumentCompressor

retriever = ContextualCompressionRetriever(
    base_compressor=RagCompressDocumentCompressor(compressor=compressor),
    base_retriever=vectorstore.as_retriever(),
)

Runnable examples: examples/basic_usage.py, examples/llamaindex_pipeline.py, examples/langchain_pipeline.py.

Benchmarks

The suite in benchmarks/ compares against no-compression, naive top-k truncation, and LLMLingua-2 on HotpotQA, TriviaQA, and Natural Questions — measuring token reduction, latency, citation validity (does the compressed context still contain a gold-supporting sentence), and downstream EM/F1 when a generator LLM is configured:

pip install ragcompress[benchmarks]
python benchmarks/run_benchmark.py --dataset hotpotqa --n 500 --generator anthropic

Sample run — HotpotQA distractor (n=30), 500-token budget, CPU (Apple Silicon), default cross-encoder scorer, retention metrics (add --generator anthropic for downstream EM/F1; llmlingua2 skipped — run with the [benchmarks] extra installed to include it):

Naive truncation at a similar budget silently drops a gold-supporting sentence in 57% of questions; ragcompress keeps it while cutting more tokens. Full report: benchmarks/results/hotpotqa_n30.md.

API sketch

@dataclass
class Chunk:
    doc_id
    chunk_id
    text
    metadata


@dataclass
class ScoredSentence:
    text
    score
    source_chunk_id
    source_doc_id
    sentence_index_in_chunk


@dataclass
class CompressedContext:
    text
    sentences
    original_token_count
    compressed_token_count
    compression_ratio

CompressedContext.provenance() returns (doc_id, chunk_id) per kept sentence; .sources() returns the contributing doc IDs.

Guarantees

• Extractiveness invariant (tested): every output sentence is a verbatim substring of some input chunk — never paraphrased or altered.
• Provenance invariant (tested): every output sentence's source_chunk_id/source_doc_id maps back to the chunk that contains it.

Development

git clone https://github.com/ragcompress/ragcompress && cd ragcompress
pip install -e ".[dev,langchain,llamaindex]"
pytest                                          # fast unit tests, no downloads
RAGCOMPRESS_RUN_MODEL_TESTS=1 pytest -m model   # real-model smoke tests
ruff check . && mypy

See CONTRIBUTING.md. Licensed under Apache-2.0.