groundrag

A production-grade advanced RAG system in Python, usable from a CLI, an HTTP API or as an MCP server (protocol revision 2026-07-28): multi-format ingestion, structure-aware chunking, hybrid dense + BM25 retrieval on LanceDB, query rewriting / multi-query / HyDE, cross-encoder reranking, cited generation with an explicit "not found" fallback, and an evaluation gate (recall@k, MRR, nDCG, faithfulness, answer relevance, abstention).

Docs: usage guide · configuration reference · design and default rationale.

Validation status. The embedding model (FastEmbed), the cross-encoder, tiktoken, Docling and MarkItDown (parsing, LLM image description, RapidOCR fallback) were exercised against real models with the opt-in suite (make test-real). The LLM paths (cited generation, query rewrite, multi-query, LLM judge) were exercised against a real OpenAI-compatible endpoint with a free reasoning model (also used for the translation evaluation, docs/DESIGN.md §6); with such models the LLM judge can return invalid JSON (faithfulness 0), so prefer eval.judge=offline or a non-reasoning model. HyDE and the OAuth / HTTP MCP paths have only been tested with stand-ins. See docs/USAGE.md §11.

Installation

pip install groundrag
# with extras (combine as needed):
pip install "groundrag[parsers,api,mcp,tiktoken,translation]"

The GitHub repository stays private until December 2026: until then the git form below and the repository links in this README work only for collaborators.

Or straight from GitHub:

pip install "git+https://github.com/andrea9293/groundrag.git"
pip install "groundrag[parsers,api,mcp,tiktoken,translation] @ git+https://github.com/andrea9293/groundrag.git"

Development install:

git clone https://github.com/andrea9293/groundrag.git && cd groundrag
make venv install          # python3 -m venv .venv && pip install -e ".[dev]"

The repository's config.yaml is picked up from the current directory. To keep personal settings (endpoint, model, paths) out of git, copy it to a gitignored config.local.yaml and run GROUNDRAG_CONFIG=config.local.yaml groundrag ... (or --config config.local.yaml).

Quickstart

The make targets below need a clone of the repository (they use data/sample/, config.yaml and the Makefile). If you installed from PyPI, skip to the groundrag ... commands in the next block.

make ingest                # index data/sample (16 file types) into collection "default"
make query Q="What is the warranty period of the RX-200?"
make demo                  # ingest + questions answered from pdf, docx, xlsx, rst (+ a "not found")
make test                  # pytest (offline, deterministic stand-ins)
make test-real             # opt-in: real embedding model, cross-encoder, LLM (network)
make eval                  # gold set → metrics → reports/eval_latest.json; non-zero exit on regression
make serve                 # HTTP API (pip install -e ".[api]")
make mcp                   # MCP server, stdio (pip install -e ".[mcp]") — see docs/USAGE.md §8

For layout-aware parsing and image description (hybrid Docling + MarkItDown): pip install -e ".[parsers]" (see docs/USAGE.md §5).

With an OpenAI-compatible LLM: export OPENAI_API_KEY=... (or set llm.base_url for Ollama / vLLM). Without one, the pipeline runs fully offline (see Limitations).

Quick start commands (your own documents; these work with pip install groundrag):

# 1. index your documents into a collection (incremental: unchanged files are skipped)
groundrag ingest docs/ --collection handbook

# 2. ask a question, offline: no LLM, the answer quotes the best sentences
groundrag --set llm.backend=none query "How many vacation days do I get?" --collection handbook

# 3. ask with an LLM (cited answer, refuses when the documents don't say)
#    the key comes from the environment (never commit it); --set llm.base_url=... / llm.model=... pick another endpoint
export OPENAI_API_KEY=...
groundrag query "How many vacation days do I get?" --collection handbook

# 4. create a gold set with an LLM, review it, then set the collection's threshold
groundrag gold-generate --collection handbook --out data/eval/gold_handbook.jsonl
# dry run (nothing written), then --save
groundrag calibrate --collection handbook --gold data/eval/gold_handbook.jsonl
groundrag calibrate --collection handbook --gold data/eval/gold_handbook.jsonl --save

# 5. measure quality, inspect, clean up
groundrag eval --gold data/eval/gold_handbook.jsonl --collection handbook --no-ingest
groundrag stats                                    # collections, rows, calibration state
groundrag drop --collection handbook               # delete a collection (asks to confirm)

More CLI examples:

# these queries use the LLM: run the export from step 3 first
groundrag query "vacation days" --filter format=docx --filter date_from=2026-01-01
groundrag query "salary band?" --groups hr-admins
groundrag query "and the RX-400?" --history chat.json --json
groundrag --set rerank.top_k=8 query "..."                   # override any config key
groundrag ingest docs/ --collection handbook --prune         # incremental; --rebuild to reset

Architecture

ingest ─► clean/dedup ─► translate to English (when active; keeps original) ─► chunk (recursive|semantic|parent-child, metadata) ─► embed (normalised)
       ─► LanceDB: vectors (flat→HNSW) + BM25 FTS (+ BM25 on original_text) + scalar/ACL columns + manifest
query ─► cleanup ─► translate to English (when active) ─► standalone rewrite ─► multi-query / HyDE ─► routing
      ─► recall: dense + BM25 per variant, pre-filtered ─► RRF (k=60) ─► top-50
      ─► rerank: cross-encoder ─► top-5 ─► score < min_score ? "not found"
      ─► context "[n] source — page N" (token budget) ─► generate (T=0.1, cited, 1 retry)
eval  ─► gold.jsonl ─► retrieval + answer metrics ─► JSON report + RAGAS JSONL ─► gate

Every log line is JSON with trace_id and per-stage timings_ms.

Agent skills

The repository ships Agent Skills in skills/ that teach a coding agent how to use groundrag: groundrag-cli, groundrag-python-integration, groundrag-ingest-parsers, groundrag-tuning and groundrag-evaluation. Install them with:

npx skills add andrea9293/groundrag                       # pick the skills interactively
npx skills add andrea9293/groundrag --skill groundrag-cli    # or a single one

Configuration

Full per-key reference (allowed values, defaults, effects): docs/CONFIG.md. Most important keys (code defaults; config.yaml has the same values except where noted):

Justifications for the defaults: docs/DESIGN.md §2.

Limitations

• Validation scope. HyDE and the OAuth / HTTP MCP paths have only been tested with stand-ins; the LLM judge can fail with reasoning models. See the validation status above.
• Offline fallbacks. Without access to HuggingFace / an LLM API the auto backends use a hashing embedder, a lexical reranker and an extractive answerer (all logged as WARNING). They're deterministic and pass the eval gate on the sample set, but they don't understand synonyms (e.g. "EU" vs "European Union"). For production quality, use FastEmbed models and an LLM.
• The token count is a regex estimate unless tokens.counter: tiktoken. The contextual header (title > section) sits on top of chunk_size.
• Translation. Translation quality is that of the NMT model: tables, names and short queries can come out wrong, which cost a few answers on same-language Italian questions in the evaluation (one run, 13 questions). Only English is a target language. Routing keywords must be in English while translation is active. The first ingest after translation becomes active re-chunks the documents it sees; unseen ones need --rebuild or --prune.
• The BM25 index is rebuilt after each ingest batch, which is fine up to roughly 10⁵ chunks. For larger corpora, batch your ingests.
• The HTTP API has no authentication. Run it behind a gateway. The MCP server over HTTP requires OAuth (or an explicit opt-out for use behind a gateway).
• The MCP Tasks extension is not yet supported by the Python SDK, so MCP ingest_documents is synchronous (and disabled by default).
• The gold set is small (23 questions: 20 answerable and 3 unanswerable traps) and synthetic. Extend it with real questions before relying on it for tuning.
• "Not found" thresholds. Without an LLM no threshold can reject questions that are on topic but have no answer in the documents (the cross-encoder scores them ~0.98); only an LLM does that. groundrag calibrate reports how many it would reject (trap_rejection) instead of hiding it. With few gold questions (< 30 answerable) the threshold is flagged few_data.
• Generated gold sets (groundrag gold-generate) are easier than real questions and are marked needs_review. Use them as a starting point and recalibrate with real questions.