The basics, in four pictures

1 · RAG — don't ask the model what it remembers; retrieve first, answer from the retrieved text only:

flowchart LR
    Q(["❓ Question"]) --> R["🔎 Retrieve the few<br/>relevant passages"]
    K[("📚 Your papers")] --> R
    R --> G["🤖 Local LLM answers from<br/>those passages only"]
    G --> A(["✅ Answer with citations<br/>you can check"])

2 · Hybrid retrieval — two complementary signals, merged by rank position (no score calibration):

flowchart LR
    Q(["Question"]) --> D["🧠 Dense — BGE-small<br/>matches meaning"]
    Q --> S["🔤 Sparse — BM25<br/>matches exact words"]
    D --> F["⚖️ Reciprocal rank fusion<br/>k = 60"]
    S --> F
    F --> T(["Top-k passages"])

3 · Corrective retrieval — pay for a second pass only when the first one looks weak:

flowchart LR
    P1["First pass"] --> C{"top dense cosine<br/>≥ τ = 0.55?"}
    C -- yes --> OUT(["use results"])
    C -- no --> X["expand query with top<br/>terms from pass 1"] --> P2["retrieve again"] --> FU["fuse both passes"] --> OUT

4 · Verification — a citation that exists is not a citation that is true:

flowchart LR
    DA["Draft answer with<br/>citation markers"] --> V1{"marker points to<br/>a real source?"}
    V1 -- no --> ST["strip it + warn"] --> V2
    V1 -- yes --> V2{"optional NLI: does the<br/>source entail the sentence?"}
    V2 --> OUT(["verified answer<br/>+ faithfulness score"])

Methodology

• Strictly additive ablation — each component is measured against the previous stack:

flowchart LR
    A["BM25"] --> B["Dense"] --> C["Hybrid<br/>(RRF)"] --> D["+ Adaptive<br/>weights"] --> E["+ Corrective<br/>PRF"]

• Two BEIR datasets — SciFact (300 claims / 5,183 abstracts) · NFCorpus (323 queries / 3,633 docs)
• Deterministic — exact search, CPU-only, seed 0; models and datasets pinned to git revisions
• Statistics, not vibes — bootstrap 95% CIs · paired randomization tests · Holm correction
• Faithfulness as evidence detection — NLI vs lexical vs cross-encoder scorers, ROC-AUC / PR-AUC
• Regression-gated — CI reruns a vendored sample; the build fails below committed floors
• One-command reproduction — make reproduce regenerates every number in this file

Results

Three findings:

• Embedder is the biggest lever — MiniLM → BGE-small: +0.060 nDCG@10 (SciFact, p < 0.001)
• Hybrid fusion beats BM25 on both datasets — +0.037 (SciFact, p = 0.002) · +0.040 (NFCorpus, p < 0.001); both 95% CIs exclude zero
• The reranker hurts here — 0.702 vs 0.734 nDCG@10 (120-query subset) at 67× the latency; reported, kept off by default

xychart-beta
    title "Retrieval cost in ms/query (120-query subset, CPU)"
    x-axis ["Adaptive hybrid", "+ Corrective PRF", "Cross-encoder rerank"]
    y-axis "ms / query" 0 --> 1400
    bar [20, 30, 1359]

Faithfulness as evidence detection (SciFact-claims dev: 188 claims, 2,031 candidate sentences, 18% positive base rate) — a plain relevance cross-encoder beats the dedicated NLI model:

xychart-beta
    title "Evidence detection, ROC-AUC (higher is better)"
    x-axis ["Lexical overlap", "NLI (DeBERTa)", "Cross-encoder"]
    y-axis "ROC-AUC" 0.6 --> 0.8
    bar [0.686, 0.688, 0.755]

Full tables with CIs and p-values: paper/main.pdf · raw per-query scores: benchmarks/results/

Architecture

flowchart LR
    subgraph Ingest
        D["PDF / DOCX / MD / TXT"] --> P[Parse] --> C["Section-aware chunks"] --> E["BGE-small embeddings"]
    end
    subgraph Index
        V[("LanceDB<br/>exact cosine")]
        B[("BM25<br/>in-memory")]
    end
    subgraph Answer
        Q(["Question"]) --> H["RRF fusion, k=60"]
        H --> G{"top dense<br/>cosine ≥ 0.55?"}
        G -- yes --> L["llama3.2:3b writes from<br/>retrieved sources only"]
        G -- no --> F["PRF: expand query,<br/>re-retrieve, fuse"] --> L
        L --> CV["Citation check:<br/>strip invalid refs"] --> N["Optional NLI<br/>grounding"] --> A(["Cited answer<br/>+ confidence"])
    end
    E --> V
    C --> B
    V --> H
    B --> H

Quality gate in CI

Every push reruns a deterministic vendored sample (50 queries, 651 abstracts, 60 claims — CPU, seed 0) via scinexusrag.eval.gate; the build fails below any floor in benchmarks/thresholds.json:

Same CI: 318 tests on Python 3.11 & 3.12 · 60% branch-coverage floor · ruff · strict mypy · gitleaks · pip-audit.

Reproduce the benchmark

Tech stack

Models & footprint

Everything is off-the-shelf and revision-pinned — nothing trained or redistributed here (PROVENANCE.md).

Runs on a laptop: ~8 GB RAM for the full stack · full ablation 15–25 min per dataset on CPU.

Quick start

pip install -e ".[eval]" && make run   # web UI + API → http://localhost:8000 (needs local Ollama)
docker compose up                      # or: containers, with a pinned Ollama service

Demo — ingest a paper, ask a question:

curl -F "file=@paper.pdf" http://localhost:8000/api/ingest
curl -X POST http://localhost:8000/api/query -H "Content-Type: application/json" \
     -d '{"question": "What is the main contribution of this paper?"}'

{
  "answer": "The paper introduces ... [1]. Experiments show ... [2]",
  "confidence": 0.82,
  "sources": [
    {"index": 1, "filename": "paper.pdf", "section_title": "Abstract", "page": 1, "score": 0.78}
  ],
  "processing_time_ms": 2140.5,
  "warnings": []
}

from scinexusrag import NexusRAG

rag = NexusRAG()
rag.ingest("paper.pdf")
result = rag.query("What did the paper find?")   # .answer, .sources, .confidence

More: notebooks/01_quickstart.ipynb · examples/

src/scinexusrag/
├── ingestion/     parser, section-aware chunker, embedder
├── retrieval/     dense, BM25, RRF hybrid, corrective PRF, reranker, SPLADE
├── generation/    Ollama client, synthesizer, citation verifier, NLI grounding
├── storage/       LanceDB vector store, document store
├── api/           FastAPI routes, security, metrics
├── eval/          datasets, metrics, systems, CI gate, reproduce
└── pipeline.py    wires it all together
benchmarks/        vendored samples, committed results, CI floors
paper/             the study (LaTeX + PDF + figures)
frontend/          static web UI

Limitations

• Two abstract-level BEIR datasets; SciFact caps at 300 queries
• Exact dense search (no ANN) — query cost grows linearly with corpus size
• BM25 index is in-memory; rebuilt on cold start
• Corrective PRF ≈ neutral on these corpora — kept: cheap, rarely fires, never regresses
• Full component-level limits: docs/ARCHITECTURE.md

Roadmap

• Datasets — FiQA, SciDocs · full-paper chunking ablations
• Models — SPECTER2 / SciNCL encoders · SPLADE, ColBERTv2, monoT5 baselines
• Scale & scoring — persistent BM25 + ANN index · RAGAs / LLM-as-judge answer scoring

Project docs

Citation

If you use NexusRAG, please cite it (CITATION.cff):

@software{bose_nexusrag_2026,
  author  = {Bose, Urme},
  title   = {NexusRAG: Local Hybrid Retrieval and Faithfulness
             Evaluation for Scientific Papers},
  version = {1.0.2},
  year    = {2026},
  url     = {https://github.com/urme-b/NexusRAG},
  license = {MIT}
}

License

MIT. Downloaded models keep their own licenses; the default generator llama3.2:3b is under the Llama 3.2 Community License (not OSI-approved, carries an acceptable-use policy).