██╗  ██╗██╗   ██╗██████╗ ██████╗ ██╗██████╗ ██████╗  █████╗  ██████╗
██║  ██║╚██╗ ██╔╝██╔══██╗██╔══██╗██║██╔══██╗██╔══██╗██╔══██╗██╔════╝
███████║ ╚████╔╝ ██████╔╝██████╔╝██║██║  ██║██████╔╝███████║██║  ███╗
██╔══██║  ╚██╔╝  ██╔══██╗██╔══██╗██║██║  ██║██╔══██╗██╔══██║██║   ██║
██║  ██║   ██║   ██████╔╝██║  ██║██║██████╔╝██║  ██║██║  ██║╚██████╔╝
╚═╝  ╚═╝   ╚═╝   ╚═════╝ ╚═╝  ╚═╝╚═╝╚═════╝ ╚═╝  ╚═╝╚═╝  ╚═╝ ╚═════╝

🎯 The Problem

┌─────────────────────────────────────────────────────────────────────────┐
│  THE FRAGMENTED WAY                                                      │
│                                                                          │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐                 │
│  │ Pinecone │  │  Neo4j   │  │  Redis   │  │ Postgres │                 │
│  │ Vectors  │  │  Graph   │  │  Cache   │  │ Metadata │                 │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘                 │
│       │             │             │             │                        │
│       └─────────────┴─────────────┴─────────────┘                        │
│                         │                                                │
│                    SYNC HELL 😱                                          │
│         If one write fails, your RAG returns                             │
│         vectors for deleted text                                         │
└─────────────────────────────────────────────────────────────────────────┘

                              VS

┌─────────────────────────────────────────────────────────────────────────┐
│  THE HYBRIDRAG WAY                                                       │
│                                                                          │
│                    ┌─────────────────────┐                               │
│                    │   MongoDB Atlas     │                               │
│                    │  ┌───┐ ┌───┐ ┌───┐  │                               │
│                    │  │ V │ │ G │ │ K │  │                               │
│                    │  └───┘ └───┘ └───┘  │                               │
│                    │  Vector Graph  KV   │                               │
│                    └─────────────────────┘                               │
│                              │                                           │
│                    ONE DOCUMENT = ATOMIC ✅                              │
│              All or nothing. Never inconsistent.                         │
└─────────────────────────────────────────────────────────────────────────┘

✨ Features

🔌 Blessed Stack

┌─────────────────────────────────────────────────────────────────────────┐
│  DATABASE         │  EMBEDDINGS       │  LLM             │  SURFACE      │
│  ────────────     │  ─────────────    │  ────────────    │  ──           │
│  ✓ MongoDB Search │  ✓ Voyage AI      │  ✓ OpenAI API    │  ✓ Python API │
│  ✓ atlas-local    │  ✓ voyage-4-large │  ✓ Compatible    │  ✓ REST API   │
│  ✓ Atlas cloud    │  ✓ Reranking      │    endpoints     │  ✓ Rich CLI   │
└─────────────────────────────────────────────────────────────────────────┘

One Supported Story

HybridRAG supports multiple integrations internally, but the public reference path is intentionally narrow:

• MongoDB: a deployment with Search and Vector Search capabilities
• Embeddings/Rerank: Voyage
• LLM: OpenAI API or an OpenAI-compatible endpoint
• Product shape: Python library + FastAPI + CLI

That is the stack exercised by the release gate. Unsupported capabilities on this path should fail explicitly rather than silently degrading.

🆕 Latest MongoDB Search

HybridRAG probes capabilities by executing them. It does not select behavior from a numeric server-version policy.

Three Filter Systems

from hybridrag import (
    # 1. Vector Search Filters (MQL - for $vectorSearch)
    VectorSearchFilterConfig,
    build_vector_search_filters,

    # 2. Atlas Search Filters (for $search compound queries)
    AtlasSearchFilterConfig,
    build_atlas_search_filters,

    # 3. Lexical Prefilters (NEW - for $search.vectorSearch)
    LexicalPrefilterConfig,
    build_lexical_prefilters,
    TextFilter, FuzzyFilter, PhraseFilter, WildcardFilter, GeoFilter,
)

Low-level lexical prefilter builders

The game-changer: Apply Atlas Search operators (fuzzy, phrase, wildcard, geo) BEFORE vector search.

from hybridrag import LexicalPrefilterConfig, HybridRAG

# Create a lexical prefilter config
filter_config = LexicalPrefilterConfig(
    # Fuzzy text matching (typo-tolerant)
    fuzzy_filters=[{"path": "content", "query": "machin lerning", "maxEdits": 2}],

    # Exact phrase matching
    phrase_filters=[{"path": "title", "query": "vector database"}],

    # Wildcard patterns
    wildcard_filters=[{"path": "tags", "query": "tech*"}],

    # Date range filtering
    range_filters={"timestamp": {"gte": "2024-01-01"}},

    # Geospatial (find docs near a location)
    geo_filters=[{"path": "location", "geometry": {"type": "Point", "coordinates": [-73.9, 40.7]}}],
)

# This is a low-level pipeline builder. HybridRAG.query() accepts the
# backend-neutral FilterConfig documented below, not lexical_filter_config.

Why Lexical Prefilters Matter

$meta Score Fields Reference

# $vectorSearch and $search use operator-specific $meta keywords.
# $rankFusion and $scoreFusion both expose the combined score via "score".
OPERATOR_SCORE_FIELDS = {
    "$vectorSearch":        "vectorSearchScore",   # legacy $vectorSearch stage
    "$search.vectorSearch": "searchScore",
    "$rankFusion":          "score",               # also "scoreDetails"
    "$scoreFusion":         "score",               # also "scoreDetails"
}

🔀 How Hybrid Search Works

  ┌─────────────────────────────────────────────────────────────────────┐
  │                    $scoreFusion (default native fusion)             │
  │                                                                      │
  │   ┌───────────────────────┐          ┌───────────────────────┐      │
  │   │    VECTOR PIPELINE    │          │    TEXT PIPELINE      │      │
  │   │  $search.vectorSearch │          │  $search.compound     │      │
  │   │                       │          │                       │      │
  │   │  ┌─────────────────┐  │          │  ┌─────────────────┐  │      │
  │   │  │ Lexical Prefilter│  │          │  │  Fuzzy Matching │  │      │
  │   │  │ (fuzzy/phrase/  │  │          │  │                 │  │      │
  │   │  │  wildcard/geo)  │  │          │  │                 │  │      │
  │   │  └────────┬────────┘  │          │  └────────┬────────┘  │      │
  │   │           ↓           │          │           ↓           │      │
  │   │  ┌─────────────────┐  │          │  ┌─────────────────┐  │      │
  │   │  │ Vector Similarity│  │          │  │  BM25 Scoring   │  │      │
  │   │  └────────┬────────┘  │          │  └────────┬────────┘  │      │
  │   └───────────┼───────────┘          └───────────┼───────────┘      │
  │               │                                  │                  │
  │               └──────────────┬───────────────────┘                  │
  │                              ↓                                      │
  │                  ┌───────────────────────┐                          │
  │                  │  Weighted Fusion      │                          │
  │                  │  vector: 0.6          │                          │
  │                  │  text:   0.4          │                          │
  │                  │  scoreDetails: true   │                          │
  │                  └───────────────────────┘                          │
  └─────────────────────────────────────────────────────────────────────┘

  ┌─────────────────────────────────────────────────────────────────────┐
  │              KNOWLEDGE GRAPH ($graphLookup)                          │
  │                                                                      │
  │   Entity Boosting: KG relationships enhance reranking scores         │
  │   Mix Mode: Combines vector + text + graph for comprehensive RAG     │
  └─────────────────────────────────────────────────────────────────────┘

🚀 Quick Start

See MongoDB value in 60 seconds (no API keys, no signup)

Prerequisites: Python 3.11+ and Docker Desktop (running).

git clone https://github.com/romiluz13/Hybrid-Search-RAG.git
cd Hybrid-Search-RAG

# Create a virtual environment and install (one-time)
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[all]"            # or just run: make first-time-setup

make demo                         # starts local MongoDB + runs the showcase

make demo brings up a local MongoDB (mongodb/mongodb-atlas-local:preview via Docker) and runs the real native hybrid-search pipeline against seeded data — no Voyage key, no LLM key, no Atlas account. You see:

• $vectorSearch — semantic nearest neighbors (cosine)
• $search — lexical / BM25
• $rankFusion — native hybrid search merging vector + lexical via RRF
• $graphLookup — knowledge-graph traversal in one pipeline

The demo uses sample vectors (labeled). In production, embeddings come from Voyage AI — that's the make demo-full path below.

Full generative RAG (bring Voyage + LLM keys)

# 1. Configure API keys
cp .env.example .env              # MONGODB_URI defaults to local; add your keys
#    Required: VOYAGE_API_KEY (https://dash.voyageai.com/)
#    Required: one LLM key — set LLM_PROVIDER to match:
#              - anthropic (default) — ANTHROPIC_API_KEY
#              - openai              — OPENAI_API_KEY
#              - gemini              — GEMINI_API_KEY
#              - grove               — GROVE_API_KEY + GROVE_BASE_URL
#                (MongoDB internal OpenAI-compatible gateway; for SAs
#                 without an OpenAI/Anthropic key)

# 2. Run full RAG (ingests real Voyage embeddings + generates an answer)
make demo-full

# 3. Or start a server
make run-api        # FastAPI  → http://localhost:8000  (/health, /docs)
make run-ui         # Chainlit → http://localhost:8001
make run-cli        # interactive CLI

macOS + Atlas SSL note: if you hit SSL: CERTIFICATE_VERIFY_FAILED against Atlas (common with python.org Python, which doesn't bundle CA certs), HybridRAG now automatically uses certifi's CA bundle for mongodb+srv:// connections. No action needed. For a corporate/custom CA, set MONGODB_TLS_CA_FILE=/path/to/your-ca.pem.

Canonical Query Paths (Python SDK)

# Simple answer
answer = await rag.query("How does hybrid search work?", mode="mix")

# Source-aware response
result = await rag.query_with_sources(
    "How does hybrid search work?",
    mode="mix",
    top_k=5,
)
print(result["answer"])
print(result["references"])

# Streaming response
stream = await rag.stream_query(
    "What should a production RAG boilerplate expose?",
    mode="mix",
    include_context=True,
    include_references=True,
)
async for chunk in stream["response_iterator"]:
    print(chunk, end="")

Release Gate

The repo is considered publish-ready only when the blessed stack passes:

• fast PR checks
• API contract tests
• example smoke tests
• deterministic live seeded validation on atlas-local:preview
• no skipped release tests
• no silent fallback on the blessed path

Basic Usage

import asyncio
from hybridrag import FilterConfig, FilterPredicate, create_hybridrag

async def main():
    # Initialize
    rag = await create_hybridrag()

    # Ingest documents
    await rag.ingest_files("./documents/")

    # Simple query
    result = await rag.query_with_memory(
        query="What are the key findings?",
        mode="mix",  # Vector + Graph + Keyword
    )
    print(result["answer"])

    # Advanced: Query with one backend-neutral metadata filter
    filter_config = FilterConfig(
        predicates=[
            FilterPredicate(
                field="metadata.category",
                operator="eq",
                value="research",
            )
        ]
    )

    result = await rag.query(
        query="machine learning trends",
        mode="naive",
        filter_config=filter_config,
    )

asyncio.run(main())

CLI

# Launch interactive CLI
hybridrag chat

# Or use Typer commands
hybridrag ingest ./documents/
hybridrag query "What is MongoDB Atlas?"
hybridrag status
hybridrag benchmark

📊 Query Modes

Supported capability surface

• Six query modes: local, global, hybrid, mix, naive, and bypass
• MongoDB score fusion by default, with rank fusion as an explicit option
• Native reranking with preserved fusion and rerank scores
• ANN and exact vector execution
• Backend-neutral metadata filters for naive retrieval
• Operator-authenticated query explanation and search-index readiness diagnostics
• Explicit all-index plan/apply/wait/rollback operations; startup does not mutate search indexes
• Optional MongoDB Automated Embedding for chunk retrieval
• Knowledge-graph entity/relationship extraction and visualization
• Conversation memory with compaction
• CLI, REST APIs, Chainlit UI, and an Ollama-compatible API
• RAGAS evaluation and Langfuse observability
• Namespace isolation, schema-validation migration, and cache maintenance tools
• Multiple production LLM providers plus engine-level bindings

Preview or beta maturity does not exclude a capability. Unsupported capabilities fail explicitly. Backup, monitoring, security, administration, and production index authorization remain deployment responsibilities.

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                              HybridRAG                                   │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────────────────┐ │
│  │   Voyage AI    │  │  Claude/GPT/   │  │      MongoDB Search        │ │
│  │   Embeddings   │  │    Gemini      │  │                            │ │
│  │   + Reranking  │  │                │  │  ┌──────────────────────┐  │ │
│  └────────────────┘  └────────────────┘  │  │ $rankFusion          │  │ │
│                                          │  │ $scoreFusion         │  │ │
│                                          │  │ $search.vectorSearch │  │ │
│                                          │  │ $graphLookup         │  │ │
│                                          │  └──────────────────────┘  │ │
│                                          └────────────────────────────┘ │
├─────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────────────────────────────────────────────────────────┐│
│  │                         FILTER SYSTEMS                              ││
│  │  Public FilterConfig → backend-specific MongoDB translators             ││
│  │  (Vector MQL)       │ (Atlas Search)       │ (Lexical prefilters)       ││
│  └─────────────────────────────────────────────────────────────────────┘│
├─────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────────────────────────────────────────────────────────┐│
│  │                         ENHANCEMENTS                                 ││
│  │  Entity Boosting │ Query Optimizer │ Self-Compacting Memory         ││
│  └─────────────────────────────────────────────────────────────────────┘│
├─────────────────────────────────────────────────────────────────────────┤
│  ┌─────────────────────────────────────────────────────────────────────┐│
│  │                        INTERFACES                                    ││
│  │        Chainlit UI  │  Typer CLI  │  REST API  │  Python SDK        ││
│  └─────────────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────────────┘

📚 Documentation

Cookbook Recipes

Architecture Decision Records

• ADR-001: MongoDB Single Database
• ADR-002: Voyage AI Embeddings
• ADR-003: Hybrid Search RRF
• ADR-004: Prompts Module
• ADR-005: Filter Builder Systems
• ADR-006: Lexical Prefilters

🧪 Development

# Setup development environment
make first-time-setup

# Run tests
make test              # All tests
make test-quick        # Fast unit tests
make test-cov          # With coverage

# Code quality
make lint              # Ruff linting
make format            # Auto-format
make typecheck         # MyPy

# Full CI suite
make ci

📊 Why MongoDB Over Postgres?

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

# Development setup
git clone https://github.com/romiluz13/Hybrid-Search-RAG.git
cd Hybrid-Search-RAG
make first-time-setup

# Run tests before submitting
make ci

📜 License

Apache License 2.0 - see LICENSE for details.

╔═══════════════════════════════════════════════════════════════════════════╗
║                                                                           ║
║   MongoDB Native: $scoreFusion • $rankFusion • $rerank • Vector Search          ║
║                                                                           ║
║   Three Filter Systems: Vector (MQL) • Atlas • Lexical Prefilters         ║
║                                                                           ║
║   One MongoDB document. Atomic updates. Never inconsistent.               ║
║                                                                           ║
╚═══════════════════════════════════════════════════════════════════════════╝

Built with MongoDB Search • Voyage AI • Claude

⬆ Back to Top