Winnow 🌾

Open-source RAG prompt compression middleware. Keep the signal. Drop the noise.

Built with FastAPI, LLMLingua-2, and tiktoken. MIT licensed, self-hostable, pip installable.

🌐 trywinnow.vercel.app · 📦 PyPI · 🤗 HuggingFace Space · ⭐ GitHub

🎯 What is Winnow?

Winnow sits between your vector database and your LLM. It takes raw retrieved document chunks, compresses them using LLMLingua-2 token-level scoring guided by your query, and returns a shorter context that preserves answer-relevant content - cutting token costs by ~50% with less than 3% accuracy loss.

✨ Key Features

• 🗜️ Token Compression: Cuts retrieved context by ~50% using LLMLingua-2
• 🎯 Query-Guided: Compression is steered by your question - relevant tokens survive
• 🔒 Protected Words: Mark phrases that must never be removed
• ⚖️ Ratio Control: Tune aggressiveness from 0.1 (light) to 0.9 (heavy)
• 🔌 OpenAI-Compatible Proxy: Drop-in /v1/chat/completions - zero code changes
• 🦜 LangChain Integration: Native WinnowRetriever drop-in wrapper
• 🐳 Self-Hostable: Single Docker command, no API key required
• 📦 Pip Installable: pip install winnow-compress

📊 Benchmarks

Tested on SQuAD with LLMLingua-2. Baseline F1: 78.4. Avg latency: ~85ms.

🚀 Quick Start

# Self-host in one command
docker run -p 8000:8000 itsaryanchauhan/winnow

API live at http://localhost:8000 · Docs at http://localhost:8000/docs

docker run -p 8000:8000 itsaryanchauhan/winnow

pip install winnow-compress

from winnow import Winnow

client = Winnow()

result = client.compress(
    text=input_text,
    compression_ratio=0.5,
    rag_mode=True,
    question="What is the warranty period?"
)

print(result["output"])
print(result["original_tokens"])     # e.g. 420
print(result["compressed_tokens"])   # e.g. 210
print(result["ratio"])               # e.g. 0.5
print(result["estimated_savings_usd"])    # e.g. 0.000525

from winnow.langchain import WinnowRetriever

retriever = WinnowRetriever(
    base_retriever,
    compression_ratio=0.5
)
docs = retriever.get_relevant_documents("your question")

curl -X POST http://localhost:8000/v1/compress \
  -H "Content-Type: application/json" \
  -d '{
    "input": "your retrieved chunks here",
    "compression_ratio": 0.5,
    "rag_mode": true,
    "question": "what is the capital of France?",
    "protected_strings": ["Paris", "France"]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": your_prompt}]
)

curl -X POST http://localhost:8000/v1/compress/batch \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": ["chunk one...", "chunk two..."],
    "compression_ratio": 0.5,
    "rag_mode": true,
    "question": "your query"
  }'

📁 Project Structure

Winnow/
├── app/               # FastAPI application
│   └── main.py        # API routes and server
├── winnow/            # pip package
│   ├── client.py      # Winnow
│   └── langchain.py   # WinnowRetriever
├── benchmarks/        # SQuAD benchmark scripts and results
├── tests/             # Test suite
├── website/           # Next.js website (trywinnow.vercel.app)
├── Dockerfile
└── pyproject.toml

🔌 API Endpoints

🛠️ Built With

• API: FastAPI + Python
• Compression: microsoft/llmlingua-2-xlm-roberta-large-meetingbank
• Tokenizer: tiktoken (cl100k_base)
• Deploy: Docker + HuggingFace Spaces

👤 Author

Created by Aryan Chauhan (@itsaryanchauhan)

📞 Get in Touch

Have questions or suggestions? Open an issue on the GitHub repository.

MIT License · Live Demo · HuggingFace Space