METEORA

METEORA selects evidence for retrieval-augmented generation (RAG): answering a question using passages retrieved from documents. It takes a query and candidate passages, then returns the passages to send to your answer-generation model.

Selection is guided by rationales: short search instructions such as “Look for clauses requiring written consent before assignment.” Rationales can be supplied by your application or generated by a language model. An optional verifier checks the selected passages against those instructions before they reach the answer-generation stage.

The package provides evidence selection, rationale generation, verification, and training utilities. You connect your own document retriever and final answer model. See the paper for the research method and benchmark results.

Installation

Requires Python 3.9 or later. The distribution and CLI are named meteora-rag; the Python module is meteora_rag.

Install from PyPI:

python -m pip install meteora-rag

To install from source instead:

git clone https://github.com/YashSaxena21/METEORA.git
cd METEORA
python -m pip install .

To install a downloaded wheel, pass its path to pip.

The base installation needs only NumPy. Optional dependencies add model support:

Extras can be combined, for example python -m pip install "meteora-rag[sentence-transformers,hf]". For installation from a source checkout, replace meteora-rag with .. Model weights are loaded separately when a model-backed component is constructed.

Quick start: find the return deadline

Imagine a customer asks: “How long do I have to return an unopened item?” Your document search has returned five store-policy passages. Only one states how long the customer has to make a return.

METEORA helps find that supporting passage in two steps:

1. A language model generates a search rationale describing the evidence to look for, such as a return deadline and any conditions on the item.
2. The selector compares that rationale with the retrieved passages and returns the matching evidence.

Install the model dependencies:

python -m pip install "meteora-rag[sentence-transformers,hf]"

The first run downloads a language model and an embedding model. This is a complete example. From a source checkout, you can also run python examples/return_policy.py.

from meteora_rag import HFRationaleGenerator, MeteoraReranker, SentenceTransformerEncoder

# The question and the passages returned by a document search.
query = "How long do I have to return an unopened item?"
documents = [
    "Unopened items can be returned within 30 days of delivery.",
    "Orders normally ship within two business days.",
    "Customer support is available Monday to Friday.",
    "Gift cards cannot be exchanged for cash.",
    "Payments can be made by credit card or bank transfer.",
]

# Show the language model the desired format using a different question.
rationale_generator = HFRationaleGenerator(
    "Qwen/Qwen2.5-0.5B-Instruct",
    sample_shots=[
        {
            "query": "When will my order arrive?",
            "response": (
                "<rationale_1>[Delivery time] Search for dispatch and delivery timelines. "
                "Flag Instructions: Reject passages that contradict the stated delivery deadline.</rationale_1>"
            ),
        }
    ],
    domain="online store policies",
    num_rationales=1,
    include_flag_instructions=True,
    use_chat_template=True,
    max_new_tokens=160,
    do_sample=False,
)

# Use semantic embeddings to compare the generated rationale with the passages.
reranker = MeteoraReranker(
    SentenceTransformerEncoder("sentence-transformers/all-MiniLM-L6-v2"),
    rationale_generator=rationale_generator,
)

# This call generates the rationale and then selects the supporting evidence.
trace = reranker.trace(query, documents)

for rationale in trace.rationales:
    print("Generated rationale:", rationale.text)
for passage in trace.documents:
    print("Selected passage:", passage)

Example output from a model run (generated wording can vary):

Generated rationale: Search for return policy details.
Selected passage: Unopened items can be returned within 30 days of delivery.

The other passages concern shipping, support hours, gift cards, and payment methods. The selected passage gives the return deadline needed to answer the customer's question: 30 days after delivery.

The sample_shots entry is a formatting example about delivery. The model creates the return-policy rationale for the current query when trace() is called. The embedding model compares that generated instruction with the five passages.

The returned passage is the evidence for answering the customer. Send trace.documents to your answer-generation model. The generated flag instructions can also be used by the optional verifier described below; this example runs rationale generation and evidence selection.

For an API example that runs without downloading model weights, see examples/minimal_selection.py.

Use METEORA in a retrieval pipeline

Pass the passages from your retriever to one of these methods:

Documents can be strings, dictionaries with text, content, or page_content, Chunk objects, or objects with a page_content or text attribute. Original document objects are preserved in filter() results.

Continuing the quick start:

print(trace.indices)
print(trace.to_dict()["selection"]["details"])

By default, results are ordered by score. Set order="document" to order them by chunk index. Selection chooses its own cutoff; top_k is an optional upper limit applied after selection and verification. It does not guarantee exactly that many results.

Without supplied rationales or a configured rationale generator, the reranker uses the query itself as one search rationale. This fallback does not call a language model. Set fallback_to_query_rationale=False to require explicit rationales or a generator.

Generate rationales with a language model

For a separate example that only generates rationales about commercial contracts, install the hf extra and run:

python examples/generate_rationales.py "Does the agreement allow termination for non-payment?"

The example uses TinyLlama/TinyLlama-1.1B-Chat-v1.0 by default. Pass --model with a different Hugging Face model ID or a local checkpoint directory. This is an example model configuration; the package does not bundle a preference-tuned METEORA checkpoint.

HFRationaleGenerator requires example query/response pairs, called sample_shots, to demonstrate the desired output format. Use examples from your domain. Each response can include a rejection rule inside the rationale block:

<rationale_1>[Termination] Search for termination clauses triggered by non-payment. Flag Instructions: Reject passages that reverse the stated termination conditions.</rationale_1>

Attach an HFRationaleGenerator as rationale_generator= when constructing a MeteoraReranker to generate rationales for each query automatically. The API guide shows a complete configuration. Flag instructions are requested by default; include_flag_instructions=False disables them.

Verify selected evidence

Verification runs only when you supply verifier=. The model receives the query, rationales, flag instructions, current chunk, and previously accepted evidence. It must return a mapping or JSON text with both boolean decisions:

{
  "relevant": true,
  "flagged": false,
  "reason": "The passage states the return window for unopened items.",
  "flag_types": []
}

This illustrates the response format. A real verifier model must decide these values for each chunk. The package accepts a chunk only when relevant is true and flagged is false. Missing or invalid decisions are rejected by default. Model connection and execution errors propagate to the caller.

A separate, complete verification example uses the bundled contract passages and a Hugging Face chat model:

python -m pip install "meteora-rag[hf]"
python examples/verify_evidence.py --model Qwen/Qwen2.5-0.5B-Instruct

The API guide explains custom model adapters, diagnostics, and verification settings. A model's judgment depends on its capabilities and the evidence supplied; verification is not an independent fact-checking service.

The small model above demonstrates the request/response interface. It can produce valid JSON while making incorrect relevance decisions. Use --model to choose a checkpoint whose decisions you have evaluated for your task.

Command-line interface

These commands run from the source checkout using the included sample files:

meteora-rag --help
meteora-rag --version

meteora-rag select \
  --chunks examples/data/chunks.json \
  --rationales examples/data/rationales.txt \
  --expansion-window 0 \
  --output outputs/selection.json

To split your own document into chunks of up to 256 words, with 25 overlapping words between adjacent chunks:

meteora-rag chunk document.txt --chunk-size 256 --overlap 25 --output outputs/chunks.json

select uses the word-hashing encoder unless --encoder-model is supplied. python -m meteora_rag exposes the same commands. See meteora-rag select --help for all options.

Train a rationale generator

DPO trains a model to prefer useful rationales over less useful ones. It requires preferred and rejected rationale examples, or candidates evaluated against known supporting passages. Raw questions alone are not sufficient training input.

The training guide describes the input schema, bundled sample data, preparation commands, training, and loading a trained checkpoint. Data preparation works with the base package; model training requires the training extra.

How selection works

The Evidence Chunk Selection Engine (ECSE) combines three operations:

1. Pair each rationale with its most similar chunk. Multiple rationales may select the same chunk.
2. Average rationale embeddings, compare the average with all chunks, and choose a cutoff from changes in similarity scores.
3. Optionally include neighboring chunks to recover nearby context.

MeteoraReranker disables neighboring expansion by default because its inputs may be independent retrieved passages. MeteoraSelector defaults to one neighbor on each side. Expansion uses input positions; use it on an ordered document or process documents separately. For very small candidate sets, adaptive pooling may retain all candidates.

The API guide documents configuration, accepted data formats, outputs, and extension points.

Examples and development

The example guide lists complete examples and their dependencies. A short Colab guide provides notebook installation instructions.

See CONTRIBUTING.md for development and tests, and PUBLISHING.md for release instructions. The Experiments/ directory contains archived research programs with separate data and environment requirements; these are not included in the installable package or release archives.

License

METEORA is licensed under CC BY 4.0. See LICENSE and ATTRIBUTION.md for attribution details. External models, datasets, and libraries retain their respective licenses.

Citation

@inproceedings{saxena2026ranking,
  title={Ranking Free {RAG}: Replacing Re-ranking with Selection in {RAG} for Sensitive Domains},
  author={Yash Saxena and Ankur Padia and Mandar S. Chaudhary and Kalpa Gunaratna and Srinivasan Parthasarathy and Manas Gaur},
  booktitle={Forty-third International Conference on Machine Learning},
  year={2026},
  url={https://arxiv.org/abs/2505.16014}
}