Sieve

  ____ ___ ______     _______ 
 / ___|_ _| ____\ \   / / ____|
 \___ \| ||  _|  \ \ / /|  _|  
  ___) | || |___  \ V / | |___ 
 |____/___|_____|  \_/  |_____|

Less noise. Better retrieval.
Sift the signal from your retrieval pipeline.

Sieve is a lightweight, post-retrieval context optimizer for Retrieval-Augmented Generation (RAG). It runs locally between your retriever and your LLM, evaluating retrieved sentences with a cross-encoder to prune irrelevant noise before generation.

Retriever ──► [ Retrieved Chunks ] ──► Sieve (Local Cross-Encoder) ──► [ Pruned Context ] ──► LLM

Your vector database over-retrieves by design: embedding top-k similarity search returns large blocks of text where often only a few sentences answer the query. Sieve removes the noise, preserves document metadata, and passes a physically smaller, denser prompt to your downstream model.

---

Why Sieve?

Every irrelevant token retrieved into your prompt creates measurable friction:

• Cost: You pay per input token on hosted APIs (OpenAI, Anthropic, Gemini, Groq).
• Latency: Prompt processing time grows with input token volume.
• Attention Degradation: Irrelevant sentences compete with crucial facts for the model's attention budget ("lost in the middle" effect).
• Zero API Key Requirement: Filtering runs locally on your machine using a compact cross-encoder (ms-marco-MiniLM-L-6-v2). Your LLM API keys are never needed for context compression.

---

Installation

Install via pip:

pip install sieve-context

Optional Extras

• LangChain Integration:

  pip install "sieve-context[langchain]"
  

• Model Context Protocol (MCP) Server:

  pip install "sieve-context[mcp]"
  

• Document Ingestion (PDF & HTML):

  pip install "sieve-context[docs]"
  

• All Capabilities:

  pip install "sieve-context[all]"
  

---

Quickstart

1. Document-Level Optimization (Preserving Metadata)

Pass retrieved documents directly into Sieve.optimize(). Sieve filters sentences while maintaining each document's identity and metadata:

from sieve import Document, Sieve

sieve = Sieve(threshold=0.5)

retrieved_docs = [
    Document(
        text=(
            "Apple Inc. reported total net sales of $391,035 million in fiscal year 2024. "
            "Apple Park's main circular building features solar panels generating 17 megawatts."
        ),
        metadata={"source": "apple-10k.pdf", "page": 28}
    ),
    Document(
        text="Steve Jobs co-founded Apple in Los Altos in 1976. iPhone net sales reached $201,183 million.",
        metadata={"source": "apple-10k.pdf", "page": 31}
    )
]

result = sieve.optimize(
    query="What was Apple's total net sales and iPhone sales in 2024?",
    documents=retrieved_docs,
)

# Ready-to-inject string for your LLM prompt
print(result.context)

# Filtered documents preserving metadata
for doc in result.documents:
    print(doc.metadata, "->", doc.text)

# Inspect exact token and reduction metrics
print(f"Original tokens: {result.metrics.original_tokens}")
print(f"Retained tokens: {result.metrics.filtered_tokens}")
print(f"Reduction: {result.metrics.reduction_percent}%")
print(f"Latency: {result.metrics.filtering_latency_ms} ms")

---

2. Direct String Filtering

If you have a raw context string:

from sieve import filter_context

result = filter_context(
    query="What was Apple's 2024 revenue?",
    context="Apple's fiscal 2024 revenue reached $391,035 million. The cafeteria serves lunch daily.",
    threshold=0.5,
    max_tokens=500,
)

print(result.filtered_context)
# Output: "Apple's fiscal 2024 revenue reached $391,035 million."

---

3. LangChain Integration

Drop Sieve into your existing LangChain retrieval chains as a document compressor:

from sieve.integrations.langchain import SieveCompressor

compressor = SieveCompressor(threshold=0.5)
optimized_docs = compressor.compress_documents(retrieved_docs, query="Apple 2024 revenue")

---

4. Model Context Protocol (MCP)

Sieve includes a native MCP server for Claude Desktop, Cursor, and any MCP-compliant agent.

Start the server:

sieve-mcp

Or configure it in your claude_desktop_config.json:

{
  "mcpServers": {
    "sieve": {
      "command": "sieve-mcp"
    }
  }
}

Available MCP Tools

1. sieve_filter:
   • Inputs: query (str), context (str), threshold (float, default 0.5), max_tokens (optional int).
   • Returns: Filtered context string, token counts, reduction percentage, and filtering latency.
2. sieve_score:
   • Inputs: query (str), context (str).
   • Returns: Unfiltered sentence-by-sentence relevance scores (0 to 1) for inspectability.
3. sieve_stats:
   • Returns cumulative session statistics: total calls, tokens processed, tokens pruned, and average reduction.

---

Sieve Calibrate

How aggressively can you filter context before answer accuracy degrades?

Sieve Calibrate sweeps a spectrum of thresholds across a benchmark dataset of {question, context, expected_answer} cases, evaluating answer preservation at each threshold to compute the minimum sufficient context.

Run calibration from the CLI:

sieve calibrate --benchmark examples/benchmark.json --thresholds 0.20,0.35,0.50,0.65,0.80 --offline

Example output:

                       SIEVE CALIBRATION RESULTS                       
+---------------------------------------------------------------------+
| Threshold         | Context Kept | Accuracy | Correct | Avg Latency |
|-------------------+--------------+----------+---------+-------------|
| — (WITHOUT SIEVE) |       100.0% |   100.0% |   10/10 |       0.0ms |
| 0.20              |        38.1% |   100.0% |   10/10 |       0.3ms |
| 0.35              |        38.1% |   100.0% |   10/10 |       0.3ms |
| 0.50              |        38.1% |   100.0% |   10/10 |       0.3ms |
| 0.65              |        38.1% |   100.0% |   10/10 |       0.3ms |
| 0.80              |        38.1% |   100.0% |   10/10 |       0.3ms |
+---------------------------------------------------------------------+

Minimum sufficient context: Threshold 0.20, 38.1% of context retained, 100.0% accuracy.

---

Configuration & Parameters

Parameter	Type	Default	Description
threshold	float	0.50	Sigmoid-squashed cross-encoder logit cutoff (0.0 to 1.0). Sentences below this score are pruned.
max_tokens	int or None	None	Optional token budget limit for the retained context.
context_window	int	0	Number of neighboring sentences within the same document to retain around high-scoring hits.
model_name	str	cross-encoder/ms-marco-MiniLM-L-6-v2	Hugging Face cross-encoder model ID.

---

Architecture & How It Works

1. Sentence Segmentation: Text is parsed into discrete sentences without external NLP runtime dependencies.
2. Local Batch Cross-Encoding: Query and sentence pairs are evaluated in parallel batches with cross-encoder/ms-marco-MiniLM-L-6-v2. The model is loaded once per process and reused.
3. Thresholding & Context Preservation: Sentences scoring above the threshold are selected. Optional context_window preserves adjacent sentences to maintain numerical and pronoun coherence.
4. Original Order Reconstruction: Retained sentences are reconstructed in their original document order to preserve narrative flow for the LLM.
5. Deterministic Metrics: Token reductions are measured locally via tiktoken (cl100k_base).

---

Limitations

• Sentence-level granularity: Sieve operates at sentence boundaries. It does not rewrite or partially redact sentences.
• Cross-Encoder Inference Time: Running a local neural model adds 50ms–300ms of local compute depending on CPU/GPU hardware.
• Non-probability scoring: Scores are sigmoid-squashed cross-encoder logits for ranking; a score of 0.8 represents ranking confidence, not an absolute statistical probability.
• Tabular Data: Regex sentence segmentation is designed for narrative prose and reports; complex ASCII/markdown tables should be handled prior to sentence chunking.

---

Contributing

Contributions are welcome! Please open an issue or pull request on GitHub:

• Repository: https://github.com/kuhinthewuh/Sieve
• Issues: https://github.com/kuhinthewuh/Sieve/issues

To set up development:

git clone https://github.com/kuhinthewuh/Sieve.git
cd Sieve
pip install -e ".[dev,all]"
pytest

---

License

This project is licensed under the MIT License.