Why adaptive-rag?

Most RAG systems have a dirty secret: they're expensive and fragile in production.

adaptive-rag is the retrieval layer that works for everyone — from a startup's first product to a hospital's air-gapped server.

⚡ 30-Second Quickstart

pip install adaptive-rag-py

from adaptive_rag import BM25Retriever, Chunk, SemanticCachedRetriever

# 1. Create a retriever — no model downloads, no API keys
backend = BM25Retriever()
backend.add([
    Chunk("policy-1", "docs", "Refunds are available within thirty days."),
    Chunk("policy-2", "docs", "Shipping takes 3-5 business days."),
    Chunk("policy-3", "docs", "Contact support at help@example.com."),
])

# 2. Wrap with semantic caching — repeated queries are free
retriever = SemanticCachedRetriever(
    backend, scope="public-docs", revision=lambda: 1
)

# 3. Search — CPU only, milliseconds latency
results = retriever.search("refund period", top_k=3)
for r in results:
    print(r.chunk.text)

🩺 RAG Doctor — Diagnose Your Retrieval in Seconds

# Diagnose any existing RAG corpus
adaptive-rag doctor --corpus ./my_docs/ --queries ./questions.json

# Or run the built-in evaluation benchmark
adaptive-rag-evaluate --repetitions 5 --output report.json

Sample output:

╔══════════════════════════════════════════════════════════╗
║         📊 RAG Health Report — my_docs/                  ║
╠══════════════════════════════════════════════════════════╣
║  Retrieval Quality:    B+  (82/100)                      ║
║  Recall@5:             0.847                             ║
║  MRR:                  0.791                             ║
║  Cache Hit Rate:       0% → 73% (with adaptive-rag)      ║
║  Estimated API Savings: ~$340/month                      ║
╠══════════════════════════════════════════════════════════╣
║  💡 Top Recommendations:                                 ║
║  1. Enable SemanticCachedRetriever (saves 73% API calls) ║
║  2. Use HybridRetriever for 12% recall improvement       ║
║  3. Adjust chunk size to 512 tokens (currently 1024)     ║
╚══════════════════════════════════════════════════════════╝

🌐 HTTP REST Server (TypeScript / Frontend / Microservices)

Serve your local document index over HTTP on CPU with zero dependencies:

# Start server
adaptive-rag serve --corpus ./my_docs/ --port 8000

Query it using standard curl or any language:

curl -X POST http://127.0.0.1:8000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "refund policy", "top_k": 3}'

Or use the zero-dependency JavaScript/TypeScript client (client-js/):

import { AdaptiveRAGClient } from "./client-js"; // or from npm package

const client = new AdaptiveRAGClient({ baseUrl: "http://127.0.0.1:8000" });
const results = await client.search("refund policy", 3);
console.log(results);

🔌 Framework Integrations

LangChain (1-line drop-in)

from adaptive_rag.integrations.langchain import AdaptiveRetriever

# Wrap your existing LangChain retriever
retriever = AdaptiveRetriever.from_corpus(docs, scope="my-app")

# Or use as a LangChain-compatible retriever directly
chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=retriever.as_langchain_retriever(),
)

LlamaIndex (1-line drop-in)

from adaptive_rag.integrations.llamaindex import AdaptiveQueryEngine

# Wrap your existing LlamaIndex index
engine = AdaptiveQueryEngine.from_index(index, scope="my-app")
response = engine.query("What is the refund policy?")

Haystack 2.x Component

from adaptive_rag.integrations.haystack import AdaptiveHaystackRetriever

retriever = AdaptiveHaystackRetriever.from_texts(texts, scope="my-app")
# Add directly to Haystack Pipeline or run standalone:
result = retriever.run(query="What is the refund policy?", top_k=3)
docs = result["documents"]

DSPy Retriever Model

from adaptive_rag.integrations.dspy import AdaptiveDSPyRetriever

retriever = AdaptiveDSPyRetriever.from_texts(texts, k=3, scope="my-app")
# Use in DSPy pipelines:
# dspy.settings.configure(rm=retriever)
passages = retriever("What is the refund policy?", k=3)

Direct use (no framework needed)

from adaptive_rag import HybridRetriever, HashingEmbedder

# BM25 + dense semantic search, CPU-only
retriever = HybridRetriever(embedder=HashingEmbedder())
retriever.add(chunks)
results = retriever.search("refund policy", top_k=5)

🏥 Domain Starter Packs

Pre-configured retrieval for specific industries — no tuning required.

# Healthcare: HIPAA-safe, no cloud, optimized for medical language
from adaptive_rag.packs import HealthcareRAG
rag = HealthcareRAG()
rag.add_documents("./patient_docs/")
results = rag.search("hypertension treatment guidelines")

# Legal: clause-aware chunking, precedent search
from adaptive_rag.packs import LegalRAG

# Education: multilingual, concept-aware
from adaptive_rag.packs import EducationRAG

# Code documentation: identifier-aware, semantic search
from adaptive_rag.packs import CodeDocsRAG

📊 Benchmark Results

Tested on the Cranfield collection — a standard IR benchmark with 1400 abstracts and 225 queries.

All benchmarks run on commodity CPU hardware. Reproduce with adaptive-rag-evaluate --dataset benchmarks/cranfield.json.

🛠️ Advanced Features

from adaptive_rag import HybridRetriever, HashingEmbedder

# Combine lexical and semantic search — no GPU required
retriever = HybridRetriever(
    embedder=HashingEmbedder(),
    anchor_boost=1.5,   # boost exact identifier matches
)
retriever.add(chunks)
results = retriever.search("machine learning optimization", top_k=10)

from adaptive_rag import MMapBM25Retriever

# Build once, read many times — survives restarts
MMapBM25Retriever.build("./my_index/", chunks)

# Open as context manager for automatic cleanup
with MMapBM25Retriever.open("./my_index/") as retriever:
    results = retriever.search("refund policy", top_k=5)

from adaptive_rag import BM25Retriever, MetadataFilteredRetriever, MetadataFilter

backend = BM25Retriever()
# Filter by department, date, user permissions, etc.
retriever = MetadataFilteredRetriever(
    backend,
    MetadataFilter(must={"department": "legal", "year": 2024})
)

from adaptive_rag import AsyncSegmentCoordinator

async with AsyncSegmentCoordinator("./index/") as coord:
    await coord.add_async(new_chunks)
    results = await coord.search_async("query", top_k=5)

📦 Installation

# Core (no dependencies)
pip install adaptive-rag-py

# With PDF support
pip install adaptive-rag-py[pdf]

# With image/OCR support (requires Tesseract)
pip install adaptive-rag-py[images]

# With NumPy SIMD acceleration
pip install adaptive-rag-py[numpy]

# Everything
pip install adaptive-rag-py[all]

🗺️ Roadmap

• BM25 retrieval (CPU-only)
• Semantic caching layer
• Hybrid BM25 + dense retrieval
• MMap disk-backed persistent index
• Built-in evaluation CLI & RAG Doctor (adaptive-rag doctor)
• Metadata filtering
• Async indexing
• LangChain adapter (adaptive_rag.integrations.langchain)
• LlamaIndex adapter (adaptive_rag.integrations.llamaindex)
• Haystack 2.x adapter (adaptive_rag.integrations.haystack)
• Domain packs: Healthcare, Legal, Education, Code Docs, Finance
• Document Loaders: PDF, OCR images, HTML, Word DOCX
• Public leaderboard & Interactive Cost Calculator (website/ and GitHub Pages)
• TypeScript/JavaScript client bindings (client-js/ package)
• Multilingual benchmark suite (benchmarks/multilingual_benchmark.json)
• DSPy framework adapter (adaptive_rag.integrations.dspy)

🤝 Contributing

We love contributions! See CONTRIBUTING.md for guidelines.

Good first issues:

• Add a new file loader (DOCX, HTML, EPUB)
• Add a new benchmark dataset
• Improve evaluation metrics
• Write integration tutorials
• Translate README to another language

💬 Community

• 💬 GitHub Discussions — Ask questions, share results
• 🐛 Issues — Bug reports and feature requests
• 📖 API Reference — Full API documentation
• 📊 Benchmarks — Benchmark datasets and results

📝 License

Apache 2.0 — see LICENSE for details.