---

doc_id: fusion-rag/readme product: fusion-rag class: platform entry: service doc_type: readme title: {zh: "Fusion-RAG 离线向量知识库后端", en: "Fusion-RAG Offline Vector Knowledge Base Backend"} audience: engineer status: draft version: 0.8.21 updated: 2026-11-07 owner: "@dahai" source_of_truth:

• README.md

---

---

Why Fusion-RAG?

Feature	Fusion-RAG	Dify RAG	LangChain RAG	Claude RAG
MLX native	✅ fusion-mlx API	❌ Ollama/Cloud	❌ Cloud API	❌ Cloud
Apple Silicon optimized	✅ LanceDB + MLX	❌	❌	❌
Multi-KB isolation	✅	✅	❌	❌
BM25 + Vector hybrid	✅ Okapi BM25 + RRF	❌	❌	❌
Contextual Retrieval	✅ Anthropic-style	❌	❌	✅
MCP Server	✅ Claude/Cursor native	❌	❌	❌
GraphRAG	✅ Entity-relation graph (/graph/* routes, experimental)	❌	❌	❌
Code AST chunking	✅ Python AST-aware	❌	❌	❌
Version snapshots	✅ Hard-link snapshots	❌	❌	❌
Incremental sync	✅ MD5 + mtime detection	❌	❌	❌
Search templates	✅ Preset + custom	❌	❌	❌
Audit logging	✅ Full search audit trail	❌	❌	❌
Fine-grained permissions	✅ ACL with role inheritance	❌	❌	❌
Cloud embed fallback	✅ Auto-fallback on local fail	❌	✅	✅
Git repo indexing	✅ Clone + .gitignore aware	❌	❌	❌
Bench API	✅ Latency benchmarks	❌	❌	❌
Distributed storage	✅ StoreBackend abstraction	❌	❌	❌
Local offline	✅ Offline by default	⚠️ Partial	❌	❌
Zero API cost	✅	❌	❌	❌

One sentence: Fusion-RAG is a unified local vector knowledge base backend for the Fusion-MLX ecosystem — all Embedding goes through fusion-mlx HTTP API, no direct MLX imports.

---

Quick Start

Prerequisites

• macOS with Apple Silicon (M1–M5)
• Python 3.12+
• fusion-mlx running on localhost:11432

Install

From PyPI (after the first tagged release is published):

pip install fusion-rag
# optional extras:
pip install "fusion-rag[qdrant]"   # Qdrant vector backend

From source (development):

git clone https://github.com/dahai80/fusion-rag.git
cd fusion-rag
pip install -e ".[test]"

Start the Server

./start.sh start
# Fusion-RAG running on http://127.0.0.1:11436

Minimal Example

import asyncio
from fusion_rag import KnowledgeBase, DocumentParser, Chunker
from fusion_rag.embed.client import EmbeddingClient
from fusion_rag.store.vector_store import VectorStore

async def main():
    # 1. Parse a document
    parser = DocumentParser()
    result = await parser.parse("README.md")
    print(f"Parsed: {result.file_name} ({result.chars} chars)")

    # 2. Chunk the text
    chunker = Chunker(strategy="semantic")
    chunks = await chunker.chunk(result)
    print(f"Chunks: {len(chunks)}")

    # 3. Embed via fusion-mlx
    embed = EmbeddingClient(model="BGE-M3")
    vectors = await embed.embed_batch([c.text for c in chunks])
    print(f"Vectors: {len(vectors)}")

asyncio.run(main())

---

API Reference

Fusion-RAG provides a REST API at /kb/* for knowledge base operations.

Knowledge Base Management

Method	Endpoint	Description
GET	/kb/bases	List all knowledge bases
POST	/kb/bases	Create a knowledge base
GET	/kb/bases/{id}	Get knowledge base details
DELETE	/kb/bases/{id}	Delete a knowledge base
GET	/kb/bases/{id}/stats	Get KB statistics

Document Operations

Method	Endpoint	Description
POST	/kb/bases/{id}/documents	Upload and index a document
POST	/kb/bases/{id}/documents/batch	Batch upload multiple documents
POST	/kb/bases/{id}/documents/ingest	Ingest inline content: text-ish types (text/markdown/html/csv) index directly; pdf/docx/epub treat content as base64-encoded binary and run the full parse pipeline (incl. the scanned-PDF OCR path) — invalid base64 returns 400 (issue #90)
GET	/kb/bases/{id}/documents	List all documents in a KB
PATCH	/kb/bases/{id}/documents/{doc_id}	Patch document lifecycle metadata (status: active/deprecated, superseded_by) — issue #86
DELETE	/kb/bases/{id}/documents/{doc_id}	Delete a document and its chunks
PUT	/kb/bases/{id}/documents/{doc_id}	Replace a document (re-index)
GET	/kb/bases/{id}/documents/{doc_id}/status	Get document indexing status
POST	/kb/bases/{id}/scan	Scan directory and index all files
POST	/kb/bases/{id}/watch	Watch files for changes
POST	/kb/bases/{id}/unwatch	Stop watching files
GET	/kb/bases/{id}/watch/status	Get watch status

Search & RAG

Method	Endpoint	Description
POST	/kb/bases/{id}/search	Semantic search (hybrid, rerank, templates, status_filter)
POST	/kb/bases/{id}/ask	RAG Q&A (model, hybrid, rerank, folder_prefix, status_filter, citations)
POST	/kb/bases/{id}/similar	Near-duplicate / version-drift search by text or raw embedding (issue #83)

Search Parameters

Parameter	Default	Description
query	(required)	Search query text
top_k	10	Number of results
threshold	0.0	Minimum similarity score
hybrid	false	Enable BM25+Vector hybrid search
hybrid_alpha	0.7	Vector weight (alpha fusion)
hybrid_method	"rrf"	Fusion method: "alpha" or "rrf"
rerank	false	Enable reranking (default ON when FUSION_RAG_RERANK_MODEL is set)
rerank_backend	from env	Rerank stage: cross_encoder (fusion-mlx /v1/rerank, real bge-reranker-v2-m3) or llm (legacy prompt-scoring)
rerank_model	from env	Cross-encoder model name (overrides FUSION_RAG_RERANK_MODEL)
rerank_top_n	20	Candidate pool size fed to the reranker before truncating to top_k (recall lift)
folder_prefix	(none)	Filter results by folder path prefix
template	(none)	Search template name (general/code/design)
rewrite_mode	(none)	Query rewrite: hyde, expand, condense
status_filter	"active"	Document status filter: active (default — drops deprecated docs), deprecated, all (issue #86)

Ask Parameters

Parameter	Default	Description
question	(required)	Question text
model	"qwen3.5-9b"	LLM model for answer generation
max_tokens	4096	Max output tokens
temperature	0.3	Sampling temperature
hybrid	false	Enable hybrid search
rerank	false	Enable reranking (default ON when FUSION_RAG_RERANK_MODEL is set)
rerank_backend	from env	cross_encoder (fusion-mlx /v1/rerank) or llm (legacy)
rerank_model	from env	Cross-encoder model name
rerank_top_n	20	Candidate pool size before truncating to top_k
folder_prefix	(none)	Filter by folder path
status_filter	"active"	Document status filter: active, deprecated, all (issue #86). Deprecated chunks kept via all are marked [已作废版本] in the LLM context; the response carries per-chunk citations[] + has_low_confidence (issue #85)

Version Snapshots

Method	Endpoint	Description
POST	/kb/bases/{id}/versions	Create a snapshot
GET	/kb/bases/{id}/versions	List snapshots
GET	/kb/bases/{id}/versions/{vid}	Get snapshot details
POST	/kb/bases/{id}/versions/{vid}/rollback	Rollback to snapshot
DELETE	/kb/bases/{id}/versions/{vid}	Delete a snapshot

Search Templates

Method	Endpoint	Description
GET	/kb/bases/{id}/templates	List search templates
POST	/kb/bases/{id}/templates	Create a custom template
DELETE	/kb/bases/{id}/templates/{name}	Delete a custom template

Permissions

Method	Endpoint	Description
GET	/kb/bases/{id}/permissions	List permission rules
POST	/kb/bases/{id}/permissions	Add a permission rule
DELETE	/kb/bases/{id}/permissions/{rule_id}	Delete a permission rule
POST	/kb/bases/{id}/permissions/check	Check if action is allowed

Audit & Admin

Method	Endpoint	Description
GET	/kb/bases/{id}/audit	Query audit logs
GET	/kb/bases/{id}/audit/{log_id}	Get a specific log entry
GET	/kb/bases/{id}/audit/export	Export audit logs (json/csv)
POST	/kb/bases/{id}/sync	Incremental directory sync
POST	/kb/bases/{id}/bench	Run search benchmark
GET	/kb/bases/{id}/bench/results	List benchmark results
GET	/kb/config	View runtime config (env-driven knobs, read-only)

Project-KB Mapping

Method	Endpoint	Description
POST	/kb/projects/{project_id}/kb	Map project to KB
GET	/kb/projects/{project_id}/kb	Get project KB mapping
DELETE	/kb/projects/{project_id}/kb	Remove project KB mapping

MCP Protocol

Method	Endpoint	Description
POST	/mcp	MCP JSON-RPC handler (tools/list, tools/call)

System

Method	Endpoint	Description
GET	/kb/status	Service status
GET	/health	Health check
GET	/metrics	Prometheus metrics (RED: request count, latency histogram, error count by endpoint + kb_id)

BNUP Textbook Corpus (Issue #74)

BNUP (北师大版) K-12 math textbook corpus ingestion + retrieval, serving synthetic training-data generation for fusion-trainer. The corpus JSON (beishi-math-g1-6.json, grades → units → lessons → knowledge_point_ids) is POSTed by the operator; fusion-rag tags each lesson with edition=bnup, grade, semester, unit, knowledge_point_ids, visual_type and indexes it into a dedicated KB for hybrid retrieval.

Method	Endpoint	Description
POST	/api/v1/bnup/ingest	Ingest the BNUP corpus JSON body into the default KB (bnup-default, or ?kb_id=). Idempotent — re-ingest overwrites. verify_api_key gated.
GET	/api/v1/bnup/retrieve	Retrieve lessons by ?knowledge_point= and/or ?grade= (&limit=, default 10). Hybrid vector + BM25 search filtered by KG metadata.
GET	/api/v1/bnup/knowledge_graph	Export the KG hierarchy (grades → units → lessons → knowledge_point_ids) + a flat knowledge_points index for the trainer.
GET	/api/v1/bnup/misconceptions	Retrieve common-error data by ?knowledge_point= for DPO negative samples. Forward-compatible — returns empty until upstream fusion-k12-teacher #21 populates common_misconceptions; re-ingest then serves real data.
GET	/api/v1/bnup/stats	Corpus coverage per grade / semester / knowledge_point.

Example:

# Ingest the corpus (operator POSTs the JSON from fusion-k12-teacher)
curl -X POST http://127.0.0.1:11436/api/v1/bnup/ingest \
  -H "Content-Type: application/json" \
  -d @beishi-math-g1-6.json

# Retrieve lessons for a knowledge point
curl "http://127.0.0.1:11436/api/v1/bnup/retrieve?knowledge_point=math-g1-na-01&limit=5"

# Export the knowledge graph
curl "http://127.0.0.1:11436/api/v1/bnup/knowledge_graph"

PDF/DOCX teacher manuals (菁优网/高思题集) can be ingested into the same BNUP KB via the standard /kb/bases/bnup-default/docs/* endpoints with metadata={"edition":"bnup", ...} tags.

Evidence Metadata Contract (Issue #77)

Every retrieved chunk carries stable evidence anchors so consumers (e.g. EnviPermit) can bind extracted values back to their source location:

Field	Where	Description
chunk_id	top level of each /search result + /ask source	Stable chunk identifier ({doc_id}_{index}), same as id.
doc_id	top level + chunk metadata	Document identifier returned at ingest time. Falls back to the chunk-id prefix for data ingested before v0.8.5.
page	top level + chunk metadata	1-based source page for PDF chunks (per-page char spans tracked through parse → chunk). 0 = no page info. Non-PDF sources: pass page in the ingest metadata (see OCR below).
chunk_index	chunk metadata	0-based chunk position within its document.

PDF files ingested via POST /kb/bases/{kb_id}/documents get per-chunk page automatically — the parser records per-page char spans and the chunker maps each chunk's start offset back to its page.

OCR pre-extracted text: ingest one page per call via POST /kb/bases/{kb_id}/documents/ingest with the caller's own page number in the metadata:

curl -X POST http://127.0.0.1:11436/kb/bases/{kb_id}/documents/ingest \
  -H "Content-Type: application/json" \
  -d '{"content": "<OCR-extracted text for page 3>",
       "doc_name": "permit_2024_p3.txt",
       "metadata": {"source": "ocr", "page": 3},
       "contextualize": false}'

The full metadata dict is stamped on the document AND every chunk, so metadata.source=ocr and metadata.page=3 round-trip through /search and are filterable via the filter parameter.

Excel / tables: there is no native table-chunk type. Convert spreadsheets to text pre-ingest (e.g. via fusion-office) and ingest the synthetic text chunks through documents/ingest with caller-defined metadata (source=excel, sheet name, etc.).

Per-project KB pattern (~100 concurrent projects, EnviPermit-style): one KB per project via POST /kb/bases (kb_id=project_<id>), ingest project documents, search with POST /kb/bases/project_<id>/search. For multi-tenant isolation enable FUSION_RAG_REQUIRE_GATEWAY=1 (KB-level tenant scoping) or FUSION_RAG_REQUIRE_IDENTITY=1 (JWT-verified tenant via fusion-identity); with FUSION_RAG_STORE_BACKEND=qdrant, tenants get separate vector collections. fusion-rag is single-process per stores dir (H3) — scale horizontally behind a stateless load balancer, never by running multiple processes against one FUSION_RAG_STORES_DIR.

Horizontal scaling + auth (issue #102): the "stateless LB" contract covers the vector path, NOT the local API-key store. auth.db is a per-instance SQLite file (~/.fusion-rag/auth.db) — a key provisioned on node A does not exist on node B. Decision (option 1, per architecture/fusion-db-conventions-1030.md DB-5): horizontally-scaled deployments use config-plane credentials only — the shared FUSION_RAG_API_KEY env on every node, or gateway/identity-issued credentials (FUSION_RAG_REQUIRE_GATEWAY=1 / FUSION_RAG_REQUIRE_IDENTITY=1). When either scaled mode is on, POST /kb/auth/keys refuses with 409 ("disabled in scaled mode") so no one can mint a single-node key; read-only key listing stays available per instance. Moving auth.db to Postgres (option 2) was rejected — it would add a PG dependency where the existing gateway/identity auth paths already cover the scaled case.

GraphRAG (Issue #79)

Entity-relation graph retrieval, wired to HTTP (experimental). A build extracts entities + relations from a KB's chunks via the fusion-mlx LLM (one call per chunk, semaphore-bounded, capped) and stores them in a per-KB SQLite graph (graph.db under the KB's store dir); graph search expands entity matches over relation hops and resolves hit chunk ids back to text.

Method	Endpoint	Description
POST	/kb/bases/{kb_id}/graph/build	Extract entities/relations from the KB's chunks (LLM-backed). max_chunks (default 200, hard cap 1000) bounds the LLM spend. verify_kb_action("write") gated.
GET	/kb/bases/{kb_id}/graph/stats	Entity + relation counts for the KB.
POST	/kb/bases/{kb_id}/graph/search	Query → entity matches → relation-hop expansion → chunks. max_hops (default 2, cap 4). Returns evidence-anchored rows (chunk_id/doc_id/page).

curl -X POST http://127.0.0.1:11436/kb/bases/{kb_id}/graph/build -H "Content-Type: application/json" -d '{"max_chunks": 200}'
curl -X POST http://127.0.0.1:11436/kb/bases/{kb_id}/graph/search -H "Content-Type: application/json" -d '{"query": "fusion"}'

Recall Benchmark (Issue #79)

scripts/recall_benchmark.py measures retrieval quality on a deterministic ≥500-chunk corpus (markdown + Python mix through the real chunker) with 55 ground-truth queries — recall@5/@10 + MRR@10, and the ablation BM25-only vs vector-only vs RRF fusion:

python scripts/recall_benchmark.py --mode mock   # wiring / no-regression evidence (labeled mode=mock)
FUSION_MLX_URL=http://127.0.0.1:11434/v1 FUSION_MLX_API_KEY=<key> \
  python scripts/recall_benchmark.py --mode real --output docs/benchmarks/recall-$(date +%F).json

Mock mode uses deterministic hashed-bag embeddings and is wiring-regression evidence only — never quality evidence. Real mode hits the live fusion-mlx embeddings endpoint with the cache disabled. CI (tests/test_recall_benchmark.py) asserts the pipeline runs and RRF fusion never drops recall@10 below the best single mode. Recorded results live in docs/benchmarks/ — see docs/benchmarks/README.md for the ablation decision.

Similar-Chunk Search (Issue #83)

POST /kb/bases/{kb_id}/similar finds near-duplicate / version-drift chunks. Input is either text (embedded with the same service-wide model as ingest, so scores are directly comparable) or a raw embedding vector:

curl -X POST http://127.0.0.1:11436/kb/bases/{kb_id}/similar \
  -H "Content-Type: application/json" \
  -d '{"text": "排污许可证 编号 X123", "top_k": 10, "threshold": 0.85, "exclude_doc_id": "<doc_id>"}'

Parameters: top_k (1-100, default 10), threshold (default: the KB's similarity_threshold), exclude_doc_id (drop a doc's own chunks — dedup-within-corpus), snippet_chars (1-2000, default 200), status_filter (active/deprecated/all). Returns matches[] with doc_id/doc_name/chunk_id/score/snippet/page/status + count + threshold.

EPUB + Scanned-PDF OCR (Issues #84/#85)

DocumentParser now reads EPUB (each XHTML chapter becomes a # <name> headed section via ebooklib). Scanned PDFs (pages present, zero extractable text) are no longer silently ingested as empty docs: they are handed to the fusion-drawing-parser OCR service over HTTP and fail visibly when none is configured:

export FUSION_RAG_OCR_URL=http://127.0.0.1:11491     # drawing-parser base URL
export FUSION_RAG_OCR_API_KEY=<jwt>                  # Bearer token (may be empty)
export FUSION_RAG_OCR_TENANT_ID=<tenant>             # X-Tenant-Id fallback (single-tenant /
                                                      # background ingestion; issue #88)

OCR chunks bypass the chunker + contextualizer (they are already slice-sized) and carry per-chunk evidence metadata (issue #85): ocr_confidence, low_confidence (vs FUSION_RAG_LOW_OCR_CONFIDENCE, default 0.85), and slice_ref (task_id/slice_id/url linking back to the source slice image). Every /search result, /ask source, and /ask citation from an OCR doc carries these fields, so consumers can flag shaky OCR text and show the original image.

Document Active/Deprecated Status (Issue #86)

PATCH /kb/bases/{kb_id}/documents/{doc_id} flips a document's lifecycle status without re-indexing:

curl -X PATCH http://127.0.0.1:11436/kb/bases/{kb_id}/documents/{doc_id} \
  -H "Content-Type: application/json" \
  -d '{"status": "deprecated", "superseded_by": "regulations_v2.pdf"}'

Status is resolved at search time from the documents table (the PATCH flips it there), so a deprecated doc vanishes from default /search, /ask, and /similar immediately. status_filter=deprecated retrieves only deprecated docs (audit/traceability); status_filter=all keeps everything — deprecated chunks then appear marked [已作废版本] in the LLM context so the model knows they are stale. /ask additionally returns per-chunk citations[] (every cited chunk, not deduped) and has_low_confidence.

---

Architecture

┌───────────────────────────────────────────────────────────────────┐
│                   Fusion-RAG HTTP API (FastAPI)                   │
│  routes_kb.py  routes_docs.py  routes_search.py  routes_admin.py │
└───────────────────────────┬───────────────────────────────────────┘
                            │
┌───────────────────────────▼───────────────────────────────────────┐
│                      RAG Core Engine                              │
│  ┌────────────────┐  ┌────────────────┐  ┌──────────────────┐   │
│  │ DocumentParser │  │    Chunker     │  │  KnowledgeBase    │   │
│  │ PDF/DOCX/MD/   │  │ semantic/fixed │  │  Manager (CRUD)   │   │
│  │ TXT/HTML/Code  │  │ /code/AST      │  │  + isolation      │   │
│  └────────┬───────┘  └───────┬────────┘  └────────┬─────────┘   │
│  ┌────────┴───────┐  ┌───────┴────────┐  ┌────────┴─────────┐   │
│  │ VersionManager │  │ SearchTemplate │  │  AuditLogger     │   │
│  │ Snapshots/Roll │  │  Manager       │  │  PermissionMgr   │   │
│  └────────────────┘  └────────────────┘  └──────────────────┘   │
└───────────┼──────────────────┼────────────────────┼──────────────┘
            │                  │                    │
┌───────────▼──────────────────▼────────────────────▼──────────────┐
│                     Storage Layer                                 │
│  ┌────────────────────────────────────────────────────────────┐  │
│  │   StoreBackend (ABC)                                       │  │
│  │   ├── LocalBackend (LanceDB + BM25)                        │  │
│  │   └── RemoteBackend (stub)                                 │  │
│  └────────────────────────────────────────────────────────────┘  │
│  ┌────────────────────────────┐  ┌────────────────────────────┐  │
│  │   MetadataStore (SQLite)   │  │  EmbeddingCache (SQLite)   │  │
│  └────────────────────────────┘  └────────────────────────────┘  │
└───────────────────────────┬──────────────────────────────────────┘
                            │
┌───────────────────────────▼──────────────────────────────────────┐
│                    Embedding Client                               │
│  Primary: fusion-mlx /v1/embeddings → Fallback: Cloud API       │
└───────────────────────────┬──────────────────────────────────────┘
                            │ HTTP
┌───────────────────────────▼──────────────────────────────────────┐
│  fusion-mlx (/v1/embeddings, /v1/chat/completions)              │
│  Apple Silicon MLX Runtime (Metal GPU)                           │
└──────────────────────────────────────────────────────────────────┘

LLM Client Infrastructure

All LLM chat calls (/v1/chat/completions) go through fusion-core's HTTP client (fusion_core.http_client):

• get_async_client(base_url, timeout) — shared connection pool (LRU, keyed by loop + base_url), reused across modules
• with_retry(fn, retries=2) — automatic retry on 429/5xx and transient connection errors (backoff + jitter)
• Auth headers passed per request, not baked into the pooled client (avoids header leak between instances sharing a base_url)
• D-H3 guard: success path checks for empty/whitespace content and degrades explicitly (logged error/fallback) rather than returning empty content as a valid result
• Pool-key timeout caveat (A7): the pool key is loop + base_url only — the per-call timeout is not part of the key, so the first caller's timeout wins for that base_url on a given loop. fusion-rag mitigates by unifying all LLM _call timeouts to a shared 60s (DocumentChain._CALL_TIMEOUT), making first-call-wins benign. Root cause tracked upstream: fusion-core#16

Modules migrated to fusion-core: contextualizer, query_rewriter, reranker, rag_chain (MultiTurnRAG + DocumentChain), graph_rag, streaming (MetadataExtractor; SSEStreamer keeps raw httpx.stream), evaluator, api/routes._generate_answer.

Key Modules

Module	File	Description
Knowledge Base	engine/knowledge_base.py	KB CRUD, config, persistence
Document Parser	engine/document.py	PDF, DOCX, EPUB, MD, TXT, HTML, code, scanned PDF via OCR service
Chunker	engine/chunker.py	Semantic, fixed, code, AST chunking
AST Chunker	engine/ast_chunker.py	Python AST-aware code chunking
BM25 Index	engine/bm25_index.py	Okapi BM25 with jieba Chinese tokenization
Contextualizer	engine/contextualizer.py	Anthropic Contextual Retrieval
Reranker	engine/reranker.py	Batch LLM reranking + HybridSearch (alpha/RRF)
Query Rewriter	engine/query_rewriter.py	HyDE, query expansion, condensation
GraphRAG	engine/graph_rag.py	Entity extraction and graph-based retrieval
Evaluator	engine/evaluator.py	RAG quality evaluation (faithfulness/relevance)
Embedding Cache	engine/embedding_cache.py	SQLite-backed embedding vector cache
Version Manager	engine/version_manager.py	KB snapshot and rollback
Incremental Sync	engine/incremental_sync.py	MD5+mtime change detection
Search Templates	engine/search_template.py	Preset + custom search strategies
Audit Logger	engine/audit_logger.py	Search audit trail with export
Trajectory Writer	engine/trajectory_writer.py	D1 retrieval trajectory sink (JSONL to ~/.fusion/trajectories/rag/)
Bench Runner	engine/bench.py	Search latency benchmarking
Permissions	permissions/acl.py	Role-based ACL with path inheritance
Git Loader	connectors/git_loader.py	Git repo clone + index
Store Backend	store/store_backend.py	Abstract storage backend
Local Backend	store/local_backend.py	LanceDB + BM25 implementation. Vector search is a brute-force scan (no ANN index) — exact results, fine up to ~100k chunks per KB; for larger corpora use the fusion-store (HNSW) or qdrant backends.
Remote Backend	store/remote_backend.py	HTTP client to a remote fusion-rag node's /store/* vector surface
Qdrant Backend	store/qdrant_backend.py	Qdrant vector DB with per-tenant collection isolation (#66)
Store Server	api/routes_store.py	/kb/bases/{kb_id}/store/* M2M vector store endpoints (RemoteBackend server half)
Auth	api/auth.py	API key authentication
MCP Server	api/mcp_server.py	Model Context Protocol for Claude/Cursor
Embedding	embed/client.py	MLX Embedding + cloud fallback
Vector Store	store/vector_store.py	StoreBackend wrapper
Metadata Store	store/metadata_store.py	SQLite document/chunk tracking
API Routes	api/routes.py	Shared helpers + sub-router mount
Server	api/server.py	FastAPI server

---

Configuration

Environment Variables

Variable	Default	Description
FUSION_RAG_PORT	11436	Server port
FUSION_RAG_HOST	127.0.0.1	Listen address
FUSION_MLX_URL	http://127.0.0.1:11432/v1	fusion-mlx URL
FUSION_MLX_API_KEY	(empty)	MLX gateway API key (auto-detected from ~/.fusion-mlx/settings.json if unset)
FUSION_RAG_EMBED	BGE-M3	Embedding model
FUSION_RAG_API_KEY	(empty)	API key auth (disabled if empty)
FUSION_RAG_AUTH_BACKEND	apikey	Auth backend: apikey or none
FUSION_RAG_INGEST_ROOTS	(empty)	Colon-separated allowed roots for file ingest/scan/watch. Empty = no confinement (local-first, unrestricted). Set to confine reads and block path traversal / LFI
FUSION_RAG_SYSTEM_PROMPT	(built-in)	Custom system prompt for RAG answer generation
FUSION_RAG_FALLBACK_URL	(empty)	Cloud embedding fallback URL
FUSION_RAG_FALLBACK_API_KEY	(empty)	Cloud fallback API key
FUSION_RAG_STORE_BACKEND	local	Vector store backend: local (LanceDB), fusion-store (HNSW via the in-tree fusion-store PyO3 binding; pip install -e ../fusion-store, not on PyPI), qdrant (Qdrant vector DB with per-tenant collection isolation; pip install 'fusion-rag[qdrant]', see QDRANT_*), or remote (HTTP client to another fusion-rag node — see FUSION_RAG_REMOTE_*)
FUSION_RAG_REMOTE_ENDPOINT	(empty)	Remote node base URL for the remote backend (e.g. http://node-b:11436). Required when backend is remote.
FUSION_RAG_REMOTE_API_KEY	(empty)	X-API-Key sent to the remote node (empty = no auth).
FUSION_RAG_REMOTE_KB_ID	(derived)	KB id on the remote node; defaults to this node's kb_id.
FUSION_RAG_REMOTE_TIMEOUT	30	Per-request timeout (seconds) for the remote backend.
QDRANT_URL	:memory:	Qdrant endpoint for the qdrant backend. :memory: = in-process local mode (no server, for dev/test); otherwise the remote Qdrant URL (e.g. http://qdrant:6333).
QDRANT_API_KEY	(empty)	API key for the remote Qdrant server (ignored in :memory: mode).
QDRANT_COLLECTION_PREFIX	tenant_id_	Prefix for per-tenant collection names. A request under tenant T lands in {prefix}{T}; no tenant (single-tenant dev) uses the shared fusion_rag_kb_{kb_id} collection.
FUSION_TRAJECTORY_DIR	~/.fusion/trajectories/rag	RAG retrieval trajectory output dir (D1 轨迹飞轮)
FUSION_RAG_SCAN_MAX_FILES	1000	Max files a single scan_directory ingests (D4)
FUSION_RAG_EMBED_CACHE_TTL	604800	Embedding cache entry TTL in seconds (D4, default 7d)
FUSION_RAG_EMBED_CACHE_MAX_ENTRIES	100000	Max rows in the embedding cache before LRU eviction (D4)
FUSION_RAG_TOKEN_BUDGET	8192	Multi-turn RAG token budget per turn (D4)
FUSION_RAG_MAX_HISTORY_TURNS	10	Max conversation turns kept in RAG history (D4)
FUSION_RAG_FETCH_K_MULTIPLIER	4	Over-fetch factor for filtered search (D4)
FUSION_RAG_RERANK_BACKEND	llm	Rerank stage: llm (legacy LLM prompt-scoring Reranker) or cross_encoder (real cross-encoder via fusion-mlx POST /v1/rerank, e.g. bge-reranker-v2-m3). Issue #70.
FUSION_RAG_RERANK_MODEL	(empty)	Cross-encoder model name for the cross_encoder backend (e.g. bge-reranker-v2-m3). When set, /search and /ask default rerank=true (rerank ON when a model is available). Empty = rerank off unless the caller passes rerank=true. Issue #70.
FUSION_RAG_RERANK_TOP_N	20	Candidate pool size fed to the reranker before truncating to top_k — recall lift without changing the returned count (issue #70).
FUSION_RAG_WATCH_CAP	16	Max concurrent directory watches per process (R3)
FUSION_RAG_TRAJECTORY_MAX_MB	100	Max trajectory file size in MB before rotation (R6)
FUSION_RAG_TRAJECTORY_KEEP	5	Rotated trajectory files to keep (R6)
FUSION_RAG_AUDIT_RETENTION_DAYS	30	Audit log retention days (0 = keep forever) (R6)
FUSION_RAG_STORES_DIR	~/.fusion-rag/stores	Per-KB store root dir (R3 watch registry)
FUSION_RAG_LOG_DIR	./logs	Log file dir (RotatingFileHandler, 10MB x 5 files)
FUSION_RAG_LOG_FORMAT	text	Log line format: text (readable) or json (one JSON object/line, for ELK/Loki)
FUSION_RAG_SKIP_STARTUP_PROBE	(empty)	Set 1 to skip the boot-time fusion-mlx reachability probe (MLX comes up later)
FUSION_REGISTRY	~/fusion/architecture/port-registry.yaml	Port registry checked by start.sh before serving (issue #98). A port missing from the registry refuses to start; FUSION_SKIP_REGISTRY_CHECK=1 bypasses; a missing registry file warns and continues (standalone deploy).
FUSION_RAG_REQUIRE_GATEWAY	(empty)	Set 1 to enforce multi-tenant isolation (#61): /kb/* requests require the X-Fusion-Route: gateway-decision header (403 without it), and KB access is scoped to the X-Fusion-Tenant header. Default off — single-tenant deployments are unaffected.
FUSION_RAG_REQUIRE_IDENTITY	(empty)	Set 1 to enable authoritative tenant resolution via fusion-identity (#68): /kb/* requests require a valid Authorization: Bearer <JWT> verified by the identity service, and the tenant is taken from the JWT tid claim. X-Fusion-Tenant is demoted to defense-in-depth (must match the JWT tid if present). Default off.
FUSION_IDENTITY_URL	http://127.0.0.1:11470	fusion-identity base URL (HTTP service, port 11470). Used only when FUSION_RAG_REQUIRE_IDENTITY=1.
FUSION_IDENTITY_SERVICE_TOKEN	(empty)	Service token sent as Authorization: Bearer to fusion-identity /api/v1/auth/verify. Required when FUSION_RAG_REQUIRE_IDENTITY=1 (fail-closed if unset).
FUSION_RAG_OCR_URL	(empty)	Base URL of the OCR service (fusion-drawing-parser, e.g. http://127.0.0.1:11491) for scanned-PDF ingestion. Empty = scanned PDFs fail visibly with "no OCR service configured" (issue #84).
FUSION_RAG_OCR_API_KEY	(empty)	Bearer token sent to the OCR service (issue #84).
FUSION_RAG_OCR_TIMEOUT	300	Total poll budget in seconds for one OCR task (issue #84).
FUSION_RAG_OCR_TENANT_ID	(empty)	X-Tenant-Id fallback for OCR calls when no request tenant is bound (background watch-loop ingestion, single-tenant deployments). The OCR service requires the header unconditionally (issue #88); during a request the calling tenant is passed through automatically.
FUSION_RAG_LOW_OCR_CONFIDENCE	0.85	OCR chunk confidence below this flags low_confidence: true on search results and /ask citations (issue #85).
FUSION_RAG_ROUTER	off	Query complexity routing (issue #128): auto = rule-based router picks simple/standard/complex per query and overrides hybrid/rerank/retry (simple -> single-path, standard -> hybrid+rerank, complex -> + 2 retry rounds). Decision logged + returned as route on /ask. Per-request override: route in {simple, standard, complex}.
FUSION_RAG_CRAG_ROUNDS	0	Bounded CRAG retry loop (issue #127): after retrieval, an LLM grader scores the chunks; fewer relevant than the threshold triggers rewrite + re-retrieve, up to this many extra rounds. Per-round trace returned as crag on /ask, rewrites chained into the citation audit. 0 = off (default). Per-request override: crag_rounds.
FUSION_RAG_CITATION_AUDIT	1	1 = write a queryable citation audit trail (issue #117, default ON since R2/#124): query chain + retrieved chunks per retrieval call to citation_audit.db, reconstructable via GET /kb/bases/{kb_id}/citations/traces. 0 = disable.

Multi-tenant isolation (#61). When FUSION_RAG_REQUIRE_GATEWAY=1, fusion-rag enforces the backend half of fusion-gateway's tenant scoping: every /kb/* request must transit the gateway (proven by the X-Fusion-Route: gateway-decision header) and carries an authoritative X-Fusion-Tenant. KBs are stamped with their tenant at creation and scoped on list/get — a tenant-A caller never sees tenant-B's KBs. The client-supplied X-Space-Id is a non-authoritative passthrough, ignored for scoping. /health, /ready, /metrics, /mcp, and auth routes stay open so the service remains observable when the gateway is down. Default off; no migration needed (existing KBs have tenant=null).

Authoritative tenant resolution (#68). The #61 path trusts the X-Fusion-Tenant header, which is forgeable by any direct-port caller. When FUSION_RAG_REQUIRE_IDENTITY=1, fusion-rag resolves the tenant authoritatively from a fusion-identity JWT instead: every /kb/* request must carry Authorization: Bearer <JWT>, which fusion-rag verifies via fusion-identity's POST /api/v1/auth/verify (service-token-gated; revocation and tenant-status enforced by the identity service). The JWT tid claim becomes the authoritative tenant and feeds the same _request_tenant contextvar, so KB scoping (#61) and per-tenant collection isolation (#66) keep working unchanged. X-Fusion-Tenant is demoted to defense-in-depth: if present it must equal the JWT tid (mismatch → 401, a blocked forgery attempt). Requests with no/invalid/revoked JWT are 401; an unreachable identity service is fail-closed (deny, not fallback to the header). fusion-identity is an HTTP service dependency (port 11470), not a Python import — fusion-rag talks to it over plain HTTP. Default off; single-tenant local-first dev is unaffected.

Using start.sh

./start.sh start      # Start the server
./start.sh stop       # Stop the server
./start.sh restart    # Restart the server
./start.sh status     # Check server status

---

Deployment

Enterprise / commercial deployment: see DEPLOYMENT.md for hard constraints (single-process, single-embedding-model, external MLX dependency), the cold-start runbook, the security checklist, the performance baseline, and the commercial-release gap tracker.

fusion-rag is a single-process service (see H3 in CLAUDE.md: do NOT run --workers N or multiple instances against one shared stores dir). Three supported deploy shapes:

Docker / docker-compose

# Build from the repo root (issue #55): the canonical Dockerfile lives at the
# repo root so the monorepo compose overlay (`build: ../fusion-rag`) resolves.
docker build -t fusion-rag:0.8.0rc4 .
docker run -p 11436:11436 \
    -e FUSION_MLX_URL=http://host.docker.internal:11432/v1 \
    -v fusion-rag-stores:/root/.fusion-rag/stores fusion-rag:0.8.0rc4
curl http://127.0.0.1:11436/health   # → 200

# Or via the compose file (builds from the root Dockerfile):
docker compose -f deploy/docker-compose.yml up -d
docker compose -f deploy/docker-compose.yml logs -f

The image uses python:3.12-slim (arm64), the Aliyun pip mirror, and binds 0.0.0.0:11436. fusion-mlx stays on the host (Apple Silicon metal), reached via FUSION_MLX_URL=http://host.docker.internal:11432/v1. fusion-rag has no fusion-memory dependency — no UDS socket mount is needed. The compose file mounts a named volume for ~/.fusion-rag/stores (KB data survives container recreation) and sets FUSION_RAG_LOG_FORMAT=json so docker logs emits parseable JSON. See the root Dockerfile + deploy/docker-compose.yml.

systemd (bare-metal / VM)

sudo cp deploy/fusion-rag.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now fusion-rag
journalctl -u fusion-rag -f

The unit drains in-flight requests for 40s on stop (TimeoutStopSec=40, matching the 30s graceful shutdown) before SIGKILL. Per-instance env overrides live in /etc/fusion-rag/env (EnvironmentFile=-). See deploy/fusion-rag.service.

start.sh (dev / single-user)

./start.sh start    # nohup, logs to logs/stdout.log (bootstrap) + logs/fusion-rag.log (rotated)
./start.sh stop     # SIGTERM, drains up to 35s, then SIGKILL

Health probes

Endpoint	Purpose	Use for
GET /health	Liveness — process up, event loop turns	restart policy (restart: on-failure, livenessProbe)
GET /ready	Readiness — embedding backend reachable + store writable	routing / load-balancer (readinessProbe); NOT restart

/health never checks downstream deps (a transient MLX hiccup must not evict the pod). /ready returns 503 with a checks body when a dep is down.

Snapshot-safe stores backup (O-P2-1)

The per-KB stores dir (~/.fusion-rag/stores/{kb}/) mixes SQLite WAL stores (metadata.db, bm25_index.db, versions.db, …) with LanceDB vectors/. A raw tar/rsync of the dir mid-write captures an inconsistent .db + a stale -wal that a restore would replay or drop. Checkpoint before backing up:

# Fold every WAL sidecar into its .db + compact LanceDB, THEN snapshot.
curl -X POST -H "Authorization: Bearer $FUSION_RAG_API_KEY" \
  http://127.0.0.1:11436/kb/bases/{kb_id}/checkpoint
tar -czf kb-snapshot.tgz -C ~/.fusion-rag/stores {kb_id}

POST /kb/bases/{kb_id}/checkpoint runs PRAGMA wal_checkpoint(TRUNCATE) on every SQLite store for the KB and table.optimize() on the LanceDB table. Run it, then snapshot.

Structured logging + request-id (O-P1-2 / O-P2-2)

• Logs rotate: logs/fusion-rag.log at 10MB x 5 files (RotatingFileHandler), so a long run no longer fills the disk. FUSION_RAG_LOG_DIR overrides the path.
• FUSION_RAG_LOG_FORMAT=json emits one JSON object per line ({ts, level, logger, msg, request_id, extra}) for ELK/Loki; text (default) stays readable for dev.
• Every HTTP request gets an X-Request-ID (incoming header honored, else generated), echoed on the response. All log lines for a request carry the same request_id, so a call is traceable end-to-end across app logs + the aggregator.

---

Data Layer Dependencies

Governing conventions: architecture/fusion-db-conventions-1030.md (single shared Qdrant instance per node; no alternative vector stores — DB-8).

When Qdrant is required

• Default (FUSION_RAG_STORE_BACKEND=local) — no external database. Vectors live in embedded LanceDB + SQLite under ~/.fusion-rag/stores/{kb_id}/.
• Qdrant backend (FUSION_RAG_STORE_BACKEND=qdrant) — required for per-tenant collection isolation (issue #66) and the multi-tenant deployments that rely on it. Install the client with pip install 'fusion-rag[qdrant]'.

Running Qdrant locally

The canonical Qdrant instance is http://localhost:6333 (HTTP); the gRPC port 6334 is reserved by the conventions doc but not used by fusion-rag — the backend connects over HTTP only. Qdrant has no native Homebrew formula, so a local run needs a container runtime (OrbStack or Docker):

docker run -d --name qdrant -p 6333:6333 -p 6334:6334 \
  -v $(pwd)/qdrant_storage:/qdrant/storage qdrant/qdrant:v1.12.4

Production provides the same instance (same ports, same version v1.12.4) via fusion-supervisor's docker-compose.sidecar.yml — do not spin up a second Qdrant.

Connection + failure behavior

Connection is configured through QDRANT_URL (per DB-3 of the conventions doc):

export FUSION_RAG_STORE_BACKEND=qdrant
export QDRANT_URL=http://localhost:6333

The default QDRANT_URL=:memory: runs an in-process ephemeral Qdrant — dev/test only, data does not survive restart.

Expected behavior when Qdrant is unreachable: ingest fails loudly — upsert/connection errors propagate to a visible 4xx/5xx, never a silently successful ingest. Known caveat (tracked as a follow-up in issue #100): the search path currently treats any get_collection failure as "collection does not exist" and returns empty results, so an unreachable Qdrant can surface as an empty search rather than an error. Check logs/fusion-rag.log for QdrantBackend errors when results look unexpectedly empty.

---

Development

# Install dev dependencies
pip install -e ".[test]"

# Run all tests
pytest tests/

# Run with coverage
pytest tests/ --cov=fusion_rag --cov-report=term-missing

# Lint
ruff check fusion_rag/

---

What's New in v0.8.41

• Permission-aware ingestion (R11, issue #133) — ingestion now captures per-document ACL metadata ({"visibility": "public"|"tenant"|"private", "tenant", "users"}) on POST /documents + /ingest and repo-level via GitLoader(acl=...); search (/search, /ask, /similar) enforces it with no caller opt-out, resolving the authoritative acl from the documents table at query time — PATCH acl (revoke = {"visibility": "public"}) propagates without reindex. Caller identity = resolved request tenant (#61/#68) + new X-Fusion-User header. Absent acl = public: ACL-less KBs and single-tenant dev unchanged; identity-less callers see public documents only.

What's New in v0.8.40

• Optional Docling layout-aware PDF ingestion (R9, issue #131) — PDF extraction was PyMuPDF-only: per-cell text with no layout understanding, so table rows scatter across column-contiguous chunks. New engine/docling_parser.py (opt-in pip install 'fusion-rag[docling]'): TableFormer ACCURATE table structure + reading order, per-page markdown export giving exact #77 page spans. Selectable per KB (pdf_backend: ""/legacy/docling) or globally (FUSION_RAG_PDF_BACKEND); ANY docling failure falls back to legacy — ingest never fails. Ablation (docs/benchmarks/docling-ablation.json): on column-major cell PDFs legacy row-intact ratio is 0.00 (no chunk holds item+value together) vs docling 1.00 with BM25 recall@10 1.0; parse cost ~2.3 s/PDF (CPU-pinned — TableFormer breaks under auto-MPS on Apple Silicon).

What's New in v0.8.39

• Bench-derived chunking profiles per KB (R8, issue #130) — chunk size/overlap were magic constants applied identically to every document type. New chunk-size ablation over the labeled corpus with real BGE-M3 + BM25 (docs/benchmarks/chunk-size-ablation.json): 512 chars @ 12.5% overlap wins recall@10 AND map@10 (768 worse on both, 1024 collapses recall to 0.5394 — topic docs merge, evidence diluted), so the default is now bench-validated. Chunker.CHUNK_PROFILES (markdown/prose 512/64, code 384/48, table 256/32) selectable per KB via chunk_profile at KB create (explicit chunk_size/chunk_overlap override; unknown profile = 400). Also fixed: KB-level chunk_overlap existed on the config but all 4 ingest Chunker call sites dropped it — every KB silently used the 64 default; now passed through. Late-chunking spike BLOCKED upstream: fusion-mlx /v1/embeddings returns pooled vectors only (dahai80/fusion-mlx#1624).

What's New in v0.8.38

• PPR graph retrieval layer (R7, issue #129) — GraphRAG.search_ppr: personalized PageRank over the extracted entity graph (query-token seeds, chunks inherit summed entity mass, ppr_score). Additive-only: fused with the vector path via RRF in the ablation; BFS path untouched; NOT wired into any route until a real multi-hop corpus validation. Ablation (docs/benchmarks/ppr-ablation.json): PPR fusion beats vector-only and vector+BFS on simple-fact (1.0 vs 0.9182) and multi-hop (0.5119 vs 0.4702) subsets; HippoRAG-2 regression check (PPR >= vector-only on simple facts) passing.

What's New in v0.8.37

• Rule-based query complexity routing (R6, issue #128) — opt-in (FUSION_RAG_ROUTER=auto or per-request route): simple queries run single-path retrieval (no hybrid/rerank/retry), standard get hybrid + rerank, complex get the full pipeline + 2 CRAG retry rounds; session history forces complex. Decision logged per query and echoed as route on /ask. Measured A/B (docs/benchmarks/router-ablation.json): routed recall@10 −0.003 vs always-full (inside gate tolerance), simple-bucket latency −32%. Default off.

What's New in v0.8.36

• Bounded CRAG retry loop (R5, issue #127) — /ask gains an opt-in grade → rewrite → re-retrieve loop (crag_rounds param / FUSION_RAG_CRAG_ROUNDS, default 0 = off): a strict LLM grader scores the retrieved chunks; below the relevance threshold the question is rewritten (QueryRewriter) and re-retrieved, results merged with dedupe, every LLM failure degrades to the current result. Per-round trace returned as crag on the answer; rewrites chain into the citation audit. Measured A/B (docs/benchmarks/README.md): retry rate 0.00 on the single-hop bench corpus (top-10 always grader-sufficient) — loop target case is weak first retrieval on multi-hop queries; re-run there before quoting lift.

What's New in v0.8.35

• Contextual BM25 ablation — context prefix lifts BM25 (R4, issue #126) — verified the production path already feeds chunk.context into the BM25 index (BM25Index.add_documents indexes context + text); new recall_bench.run_contextual_bm25_ablation + scripts/contextual_bm25_ablation.py measure the delta on the deterministic labeled corpus with the dense channel shared by both arms. Live BGE-M3 run (docs/benchmarks/contextual-bm25-ablation.json): even a crude template context (doc-id + doc opening) lifts recall@5 0.7939 → 0.8333, recall@10 0.9212 → 1.0, ndcg@10/map@10 → 1.0 — the production LLM-generated Contextualizer context is strictly richer, so enabling it is non-regressive for BM25.

What's New in v0.8.34

• Rerank pool A/B — 20 saturates (R3, issue #125) — recall_bench.run_rerank_pool_ablation + scripts/rerank_pool_ablation.py (RRF candidates per pool → cross-encoder → top-10, R12 metrics + rerank latency p50/p95). Live bge-reranker-v2-m3 run recorded (docs/benchmarks/rerank-pool-ablation.json): metrics identical at pool 20/50/100/150, latency 114 ms → 1298 ms p50 — rerank_top_n=20 default stays, re-run on a real labeled set before concluding for production traffic.

What's New in v0.8.33

• PyPI packaging — pyproject.toml PyPI-ready (readme/classifiers/urls, fusion-core>=0.12.1 resolves from PyPI); .github/workflows/pypi.yml publishes via Trusted Publishing (OIDC) on v* tags; sdist + wheel verified (console assets included). pip install fusion-rag once the first tag publishes.
• MCP kb_id optional (#136) — kb_search / kb_ask fall back to the single existing KB (tenant-scoped) or the most recently created; tools/list documents the default; the ACL gate resolves the same id so the default path cannot skip it. Typed -32602 outside app state.
• start.sh venv contract (#137) — already-activated VIRTUAL_ENV first, project-adjacent .venv second, monorepo fallback last; fails loudly when fusion_rag is not importable instead of starting uvicorn into a crash.

What's New in v0.8.32

• Eval completeness (R12, issue #134) — NDCG@K + MAP@K added alongside recall/MRR in recall_bench (all three modes) and the offline eval harness; scripts/eval_baseline.json re-baselined with ndcg@10 0.9469 / map@10 0.9212, CI gate fails on regression. Retune checklist (docs/retrieval-retune-checklist.md) + printed on gate failure: any embedding-model / chunking / fusion-parameter change requires a re-run against the fixed labeled sample. POST /search accepts source_weights (per-doc_type trust multipliers, [0,10], invalid entries ignored with a warning) — deterministic post-fusion re-rank applied before the rerank stage; scored rows carry source_weight for audit.

What's New in v0.8.31

• Citation auditing ON by default (R2, issue #124) — FUSION_RAG_CITATION_AUDIT now defaults to 1 (set 0 to disable). Measured overhead: ~0.11 ms per retrieval write (10 chunks + 1 rewrite, sqlite WAL batch) — noise against search latency. The evaluator (RAGEvaluator) gained a citation-accuracy judge (CITATION_ACCURACY_PROMPT: what fraction of the answer's claims are supported by the CITED sources — Anthropic/Glean production rubric) emitted per-case as citation_accuracy + aggregate avg_citation_accuracy (averaged only over cases that carry citations), and citation_accuracy_from_trace() scores an /ask response's citations against a citation-audit trace (chunk-id overlap).

What's New in v0.8.30

• RRF fusion consolidation (R1, issue #123) — HybridSearch(method="rrf") is the single supported fusion path; the weighted-score FusionRetriever is deprecated (DeprecationWarning, experimental only). Recall ablation re-run on live BGE-M3 recorded as docs/benchmarks/recall-2026-11-14.json (RRF 0.7939/0.9212/1.0000, rrf >= max(single) holds).

What's New in v0.8.29

• Office format ingest: PPTX + XLSX — server-side parsers (python-pptx, openpyxl): PPTX renders one ## Slide N section per slide (text frames in reading order + speaker notes; image-only slides skipped), XLSX renders one ## Sheet: name section per sheet with |-joined cell rows (cached formula values, empty trailing cells trimmed, empty sheets skipped). Both types added to the #90 inline binary ingest set and the console file picker (accept updated). Legacy binary formats (.doc/.xls/.ppt) remain unsupported — they need a converter, not a parser; re-save as docx/xlsx/pptx. New deps: python-pptx>=1.0.0, openpyxl>=3.1.0.

What's New in v0.8.28

• UI: file-picker ingest in Docs / Search tab — "选择文件入库 / Pick a file" button: pick a txt/md/markdown/html/htm/csv (text) or pdf/docx/epub (binary, #90 base64 inline path → full parse pipeline incl. scanned-PDF OCR) file; doc name = filename. The paste-text path stays as a secondary row.

What's New in v0.8.27

• UI: chat input doubled — the 问答 / Chat input is now a 2-row textarea (Enter sends, Shift+Enter newline, IME-safe), send button bottom-aligned. Console rebuilt.

What's New in v0.8.26

• UI: chat tab height doubled — the 问答 / Chat message area now scrolls within max-h-[48rem] (was max-h-96), min height raised to match. Console rebuilt.

What's New in v0.8.25

• Console chat tab — new "问答 / Chat" tab in the KB detail view: a RAG Q&A box calling POST /kb/bases/{kb_id}/ask. Answers render with per-chunk citation chips (doc name, page, score; low-OCR-confidence citations flagged ⚠ and a per-answer warning). Plain text (no markdown lib) — it's an evidence tester on top of the /ask surface. Console rebuilt (v0.8.25, bundle +1.2 kB gzip).

What's New in v0.8.24

• Citation audit trail (issue #117, feature 1) — every retrieval call (/search, /ask, /similar) can now write a queryable audit trail to citation_audit.db: one rag_queries row per query version (the original plus each rewrite step chained via parent_id/depth) and rag_citations rows for the retrieved chunks linked to the FINAL query version with per-chunk similarity + rank. An answer's citations are reconstructable from the DB alone: GET /kb/bases/{kb_id}/citations/traces (list) + GET /kb/bases/{kb_id}/citations/traces/{query_id} (full chain + evidence). Opt-in via FUSION_RAG_CITATION_AUDIT=1 (default OFF — zero behavior change); write failures log and never affect the response.
• Schema'd RAG eval set + offline eval gate (issue #117, feature 2) — engine/eval_set.py: eval_cases (query / gold chunks, seeded deterministically from the recall_bench 55-query corpus, idempotent) + eval_runs (metric rows per run). run_offline_eval() is BM25-only — no embedding model, no LLM, no network — computing recall@5 / recall@10 / mrr@10 (baseline: 0.7939 / 0.9212 / 1.0000). scripts/eval_gate.py compares against scripts/eval_baseline.json and FAILS the build when any metric drops beyond tolerance (default 0.02) while always reporting deltas; new .github/workflows/eval.yml CI job runs the gate on every PR touching retrieval code.

What's New in v0.8.23

• Docs: frontmatter fix (PR #116) — owner: @dahai quoted as owner: "@dahai" in all 15 frontmatter-bearing docs (bare @ handle is an invalid YAML mapping value in strict parsers). No runtime change.

What's New in v0.8.22

• Docs: fusion docs-system v1.1 contract (PR #115) — new docs/ tree (product page, architecture, API reference REST+MCP, config reference, get-started, backup-and-restore + multi-tenant-isolation guides, troubleshooting, benchmarks) following the platform × service contract; ci/validate_docs.py contract validator wired as a CI gate (.github/workflows/docs.yml) — sibling-link integrity + required sections enforced on every PR. Version bump only; no runtime change.

What's New in v0.8.21

• Web surface gap closed (issue #113) — the console covered 2 views over ~70 REST routes; the previously API-only capabilities now have UI. KB detail gained tabs: Graph (stats tiles, LLM entity-extraction build with explicit busy state, graph search with evidence hits), Versions (snapshot list/create, rollback + delete via destructive-action ConfirmDialog), Templates (preset + custom search templates, create/delete; builtin templates hide the doomed delete action). New #/admin page (linked from the console header) with: per-KB ACL rules (list/add/delete + live permission check), audit log (recent records + JSON/CSV export links), and API-key management (list/create/delete; a minted key is shown once). Server-side: the auth router (/kb/auth/keys) is now also mounted under the console subpath /shell/fusion-rag/ — it is registered separately from the /kb router at root level, so its endpoints 404'd behind the gateway pass-through proxy. Console calls ride page-relative paths (#109 contract preserved); bundle checks extended. 472 tests passing; ruff clean.

What's New in v0.8.20

• Console rebuilt to the union web UI spec (user report: bilingual/styling non-compliant + dead delete button) — the single-file vanilla console is replaced by a proper React 18 + TypeScript + Tailwind app (web/, Vite, Bun build) following the fusion-web-shell unified rules: design tokens via @fusion/web-ui (token-only colors, rounded-card/rounded-md, dark + light via the Shell theme bridge), bilingual copy via the SDK language bridge (t(), follows the Shell toggle), four mandatory states (loading skeleton / empty / error+retry / populated) on every data view. Delete-KB fix: the old page used native confirm(), which the shell sandboxed iframe blocks silently — the button appeared dead. Deletion now goes through ConfirmDialog (danger + type-the-KB-id guard) and feedback via the SDK toast bridge. The API-key localStorage input is removed (spec S8 — the same-origin shell session carries auth; no local token storage). Build output is committed under fusion_rag/console/ (shipped as package data, served at /shell/fusion-rag/ exactly as before); dev: cd web && bun install && bun run dev. 4 console tests (subpath parity, page-relative API base, no localStorage key, no origin-root fetch); full suite 472 passing.

What's New in v0.8.19

• Console works behind the gateway (issue #109) — the console's API calls used origin-root-relative paths (fetch("/kb/bases")); behind the gateway pass-through proxy (origin :11432, which only routes /shell/fusion-rag/*) those 404'd. The page now derives its API base from location.pathname (page-relative), and fusion-rag serves the FULL /kb router plus /health under the console subpath /shell/fusion-rag/ — registered BEFORE the StaticFiles mount so routes win over static-file matching. Direct access (:11436) unchanged; behind the gateway the browser now asks /shell/fusion-rag/kb/... and the subpath-carrying proxy forwards it to the same handlers. 4 console tests; full suite 472 passing.

What's New in v0.8.18

• Built-in RAG console (admin menu follow-up) — fusion-rag now ships a self-contained static console served at /shell/fusion-rag/ (alias /console): KB list/create/delete, inline text doc ingest + delete, hybrid search tester (top-5 with scores), live /health pill. Single HTML page owned by this repo (ships in the pip package, no build step); all API calls use relative URLs so the page works identically direct (:11436) and behind fusion-gateway (user auth already proxied via the existing rag: upstream). Gateway menu integration (manifest entry + /shell/fusion-rag/ proxy) requested upstream: dahai80/fusion-gateway#481. 3 new tests; full suite 471 passing.

What's New in v0.8.17

• Local embedding fallback: one shared load + fail-fast mode (issue #106) — when fusion-mlx was down, every embed request fell back to a local sentence-transformers BGE-M3 cold load (~2-4 min on MPS), and concurrent requests each triggered their OWN load (no lock on the module singleton) — search hung for minutes and looked hung. get_local_model now uses double-checked locking: first caller loads, concurrent callers wait and reuse (verified live: 5 fallback triggers, exactly 1 model load). New operator knob FUSION_RAG_EMBED_LOCAL_FALLBACK=0 disables the local fallback entirely: embed calls fail FAST with a retryable EmbeddingError (search returns a clean 503 with a readable detail, ~2.5s instead of minutes) instead of cold-loading a model in-request. In fail-fast mode health() no longer reports healthy via the local model, and /ready now respects health()'s return value (it previously ignored the boolean — False still reported "ok"). 8 new tests; full suite 468 passing.

What's New in v0.8.16

• Startup probe actually runs (issue #104) — the O-P2-3 startup gate called the async EmbeddingClient.health() bare: the coroutine was never awaited, so the probe passed unconditionally (a fusion-rag booted with fusion-mlx down reported healthy) and every boot emitted a RuntimeWarning: coroutine ... never awaited. The gate is now an extracted _run_startup_probe() that probes fusion-mlx GET /models directly — deliberately NOT via health(), whose local sentence-transformers fallback would mask a down MLX on boxes where the local model is installed. Boot-time console logging is configured before the probe (previously the passing probe's log line was dropped — no handler existed until run_server). MLX unreachable → loud ERROR + exit 1; FUSION_RAG_SKIP_STARTUP_PROBE=1 still bypasses. 4 new tests; full suite 460 passing.

What's New in v0.8.15

• Scaled-mode local key creation refused (issue #102) — the per-instance SQLite auth.db contradicted the documented "scale horizontally behind a stateless load balancer" contract: a key provisioned on node A 401s on node B. Decision: option 1 (config-plane keys only). When FUSION_RAG_REQUIRE_GATEWAY=1 or FUSION_RAG_REQUIRE_IDENTITY=1 is on, POST /kb/auth/keys refuses with 409 instead of minting a single-node key; scaled deployments use the shared FUSION_RAG_API_KEY env or gateway/identity-issued credentials. Single-tenant default unchanged. README documents the constraint and the rejected option 2 (auth.db to Postgres). 4 new tests; two-node drill verified.

What's New in v0.8.14

• Data layer dependencies documentation (issue #100) — new README section documenting when Qdrant is required (only for the qdrant backend / per-tenant isolation — the default local LanceDB backend needs no external DB), how to run it locally (container runtime required, qdrant/qdrant:v1.12.4), the canonical connection QDRANT_URL=http://localhost:6333 (HTTP-only — the gRPC port 6334 is reserved but unused by fusion-rag), fail-loud expectations on ingest, and a pointer to architecture/fusion-db-conventions-1030.md as the governing doc. Follow-up noted in the issue: search-path silent degradation when Qdrant is unreachable.

What's New in v0.8.13

• Config Phase 3: legacy key-name cleanup + register-then-start self-check (issue #98) — the last literal references to legacy env key names are gone from the code: key resolution reads the canonical FUSION_CLIENT_KEY only, with the legacy alias chain owned entirely by fusion_core.config.client_key() (never spelled out in this repo, so scripts/check-env.py reports 0 legacy warns for fusion-rag — and stays green when the check auto-escalates to errors on 2027-01-01). start.sh now verifies its own port-registry.yaml entry before serving (config Phase 3.3): an unregistered port fails at start with a clear message instead of at CI, FUSION_SKIP_REGISTRY_CHECK=1 bypasses, and a missing registry file warns + continues for standalone deploys. 4 new tests cover the refuse/pass/bypass/missing-file paths; full suite 473 passing.

What's New in v0.8.12

• Single-point config resolution via fusion_core (issue #96) — all hand-rolled key/env resolution is replaced by fusion_core.config (config Phase 2.2): the MLX API key resolves through client_key() (alias chain FUSION_CLIENT_KEY → FUSION_MLX_API_KEY → MLX_API_KEY; dev profile: empty + one-shot warning; FUSION_PROFILE=prod without a key: RuntimeError fail-fast), the server port through service_port("rag") (FUSION_RAG_PORT), and the data-dir root through service_data_dir (FUSION_RAG_DATA_DIR override, default stays ~/.fusion-rag) — used by the KB stores root, watch registry, auth/graph/cache/eval SQLite paths. server.py chains client_key() or resolve_api_key() so the local-first ~/.fusion-mlx/settings.json fallback keeps working; local _mlx_api_key()-style resolvers are deleted (call sites stay one-line). No env renames; FUSION_RAG_API_KEY (the service's own auth) is unchanged. 11 new tests; full suite 469 passing; scripts/check-env.py reports 0 errors for fusion-rag.

What's New in v0.8.11

• Exact chunk-id deletion (issue #94) — deleting one document no longer removes the physically-shared chunks of an identical-content doc. Two docs ingested with the same doc_path (e.g. the same inline doc_name twice) used to share one deletion key, so DELETE on either removed both docs' vectors and BM25 rows. All delete/replace/rollback paths now go through a new delete_by_chunk_ids store surface (implemented by the local, qdrant, fusion-store, and remote backends; the remote /store/* surface gained POST /delete_by_chunk_ids), with a logged delete_by_doc fallback only for old remote nodes. 5 new tests; full suite 458 passing.
• LLM auth surfacing (issue #93) — a stale FUSION_MLX_API_KEY used to break every /ask with a generic "upstream LLM unavailable". A 401/403 from fusion-mlx now surfaces distinctly: /ask returns 502 "LLM credential rejected", the startup probe logs a loud ERROR (never exits — search keeps working), and /health reports an informational llm_auth: ok/rejected/unknown field. A successful generation clears a stale rejected flag after key rotation; logs identify the key only by blake2b hash prefix. 11 new tests.
• Process title (issue #92) — the server process now shows as fusion-rag serve :11436 in ps/Activity Monitor instead of a bare Python (setproctitle, soft dependency). start.sh stops by PID file, so lifecycle control is unaffected. 3 new tests.

What's New in v0.8.10

• Inline binary ingest fix (issue #90) — POST /kb/bases/{kb_id}/documents/ingest with content_type pdf/docx/epub now treats content as base64-encoded binary: it is decoded to a temp file and run through the full DocumentParser pipeline (PDF text extraction, EPUB chapters, and the scanned-PDF OCR path from #84), instead of chunking and indexing the base64 text itself — which silently polluted retrieval with garbage chunks. Invalid base64 returns 400. The text-ish types (text/markdown/html/csv) keep the direct-text path unchanged. 5 new tests (PDF/EPUB parse round-trips, scanned-PDF OCR path, invalid base64, plain-text regression); full suite 439 passing.

What's New in v0.8.9

• OCR tenant header passthrough (issue #88) — OcrHook now sends X-Tenant-Id on every call to the drawing-parser OCR service (its tenant middleware 401s header-less calls, breaking scanned-PDF ingest in multi-tenant deployments). The tenant is resolved from the calling request's context (works with both #61 gateway and #68 identity modes); FUSION_RAG_OCR_TENANT_ID is the fallback for background watch-loop ingestion and single-tenant deployments. Invalid charset values are dropped and logged, never forwarded.

What's New in v0.8.8

• Similar-chunk search (issue #83) — new POST /kb/bases/{kb_id}/similar endpoint for near-duplicate / version-drift detection: query by text (embedded with the same service-wide model as ingest, so scores are comparable) or a raw embedding vector; top_k, threshold, exclude_doc_id (dedup-within-corpus), snippet_chars, and status_filter. Returns matches[] with full evidence anchors (doc_id/chunk_id/page/status).
• EPUB + scanned-PDF OCR (issue #84) — DocumentParser now reads EPUB (ebooklib, each XHTML chapter becomes a headed section). Scanned PDFs (pages present, zero extractable text) are handed to the fusion-drawing-parser OCR service (FUSION_RAG_OCR_URL, FUSION_RAG_OCR_API_KEY, FUSION_RAG_OCR_TIMEOUT) instead of being silently ingested as empty docs — and fail visibly when no OCR service is configured. OCR chunks bypass the chunker + contextualizer (already slice-sized) and carry per-chunk evidence metadata.
• OCR evidence metadata + citations (issue #85) — every /search result and /ask citation from an OCR-sourced chunk carries ocr_confidence, low_confidence (vs FUSION_RAG_LOW_OCR_CONFIDENCE, default 0.85), and slice_ref (task_id/slice_id/url linking back to the source slice image). /ask additionally returns per-chunk citations[] (not deduped) and a top-level has_low_confidence flag.
• Document active/deprecated status (issue #86) — PATCH /kb/bases/{kb_id}/documents/{doc_id} flips a doc's lifecycle status (active/deprecated, optional superseded_by) with no re-indexing. Status is resolved at search time from the documents table, so a deprecated doc vanishes from default /search, /ask, and /similar immediately; status_filter (active/deprecated/all) controls each query, and deprecated chunks kept via all are marked [已作废版本] in the LLM context.
• 19 new tests (EPUB parse + OCR roundtrip + failure paths, /similar validation + exclusion + status, PATCH + status filtering + /ask citations), full suite 428 green, ruff clean.

What's New in v0.8.3

• LLM backend URL + auth propagation (issue #72) — every LLM-calling engine (Contextualizer, QueryRewriter, Reranker) now honors the same FUSION_MLX_URL / FUSION_MLX_API_KEY as the EmbeddingClient instead of hardcoding the gateway URL http://127.0.0.1:11432/v1 and sending no auth header. A non-default backend (e.g. fusion-mlx directly at :11434 with a different key) no longer 401s on every chat-completions / context-generation call. The route layer passes embed.base_url + embed.api_key into each engine it constructs; start.sh now activates the monorepo .venv before launching (was bare python3, failing with ModuleNotFoundError: No module named 'fusion_core' outside an activated venv). 4 new regression tests lock the URL + Authorization header wiring.

What's New in v0.8.2

• Cross-encoder rerank (issue #70) — real cross-encoder reranking via fusion-mlx's Cohere/Jina-compatible POST /v1/rerank endpoint, replacing the legacy LLM prompt-scoring Reranker as the precision stage. New CrossEncoderReranker (engine/cross_encoder_reranker.py) is a plain async HTTP client (no MLX import — respects the project's "no direct MLX imports" + "only modify own project" rules) to fusion-mlx's rerank route, e.g. bge-reranker-v2-m3. Select it with FUSION_RAG_RERANK_BACKEND=cross_encoder + FUSION_RAG_RERANK_MODEL=bge-reranker-v2-m3; when FUSION_RAG_RERANK_MODEL is set, /search and /ask default rerank=true (rerank ON when a model is available — acceptance 1). rerank_top_n (default 20) fetches a wider candidate pool that the cross-encoder re-scores before truncating to top_k, lifting recall without changing the returned count. Fallback chain: cross-encoder → legacy LLM-prompt Reranker → original retrieval order (logged, never crashes — backward compat, acceptance 4). Hybrid retrieval (BM25 + vector + RRF fusion) was already present (hybrid=true, acceptance 2). A recall benchmark (scripts/recall_benchmark.py) runs offline (mocked embed + mocked cross-encoder) and guards the no-regression contract (rerank ≥ hybrid ≥ vector); for the absolute recall-lift magnitude, run it against live fusion-mlx with BGE-M3 + bge-reranker-v2-m3 loaded (acceptance 3). 10 new tests (unit: reorder/score-stamp/empty/truncation/404/network; integration: backend selection, fallback chain, default-off regression guard, default-on when model set, explicit rerank=true).

What's New in v0.7.2

• Audit P4 + architecture hardwounds (Task #10-18) — maintainability fixes and 硬伤 A-F 收口 from the adversarial audit (audit/fusion-rag-audit-0827.md):
  • 硬伤 B 收口 — the three-store ingest write (vectors / bm25 / metadata) now writes the hard-undo side first: vectors → bm25 → metadata via a single _write_doc_to_stores helper shared by all four ingest paths (_index_document, ingest_content, replace_document, scan_directory). A crash now leaves recoverable orphan vectors (re-ingest over doc_path) instead of orphan metadata that makes list_documents lie. Unified rollback across all paths (P4-1).
  • 硬伤 C 收口 — auth→identity→ACL wired: require_kb_action chains verify_api_key (resolves subject) → PermissionManager.check_permission, enforced on every /bases/{kb_id}/* route including reads (P0-5/6/7 already landed; this PR confirms the full chain).
  • P4 maintainability — BM25Index._init_db no longer double-opens a SQLite connection (P4-2); EmbeddingCache eviction uses a created_at-index subquery instead of an O(n) OFFSET scan (P4-3), and _hash switches MD5→blake2b and drops the 1000-char text truncation that made the stored column disagree with the hash key (P4-4); VectorStore routes every backend through StoreBackendFactory uniformly (no more hardcoded 3-branch), backends accept **_ to ignore irrelevant kwargs (P4-5); SearchTemplateManager seeds builtins once per DB (not per construction) and uses ON CONFLICT DO UPDATE so a new release's builtin defaults actually apply (P4-6); KnowledgeBase/KnowledgeBaseConfig.from_dict warn on unknown keys and raise on a load-time empty id instead of silently regenerating one (which orphaned storage) (P4-7); validate_identifier dead code removed and template validation now checks name charset/length, description cap, and doc_type_filter items (P4-8).
  • 硬伤 D 部分收口 — fusion-store backend uses score = 1/(1+dist) + threshold tuning (P0-10); the root fix (exposing Store.open(metric=...) in the upstream fusion-store binding) is tracked as an upstream issue. Default local backend (LanceDB cosine) is unaffected and recommended for production.
• 243 tests green (random + ordered), ruff clean.

What's New in v0.7.1

• Audit P2/P3 fixes (Task #8) — architecture + performance hardening from the adversarial audit (audit/fusion-rag-audit-0827.md): LocalBackend.close() now actually releases the BM25 SQLite handle (was only nulling refs); FusionStoreBackend.close() raises on checkpoint failure instead of silently losing data; EmbeddingCache inherits SqliteBase (no more connection-per-call storm) with a batched WHERE IN lookup; MetadataStore splits read/write lock paths for concurrent reads; BenchRunner/SearchTemplateManager/AuditLogger adopt the SqliteBase write lock (no more cross-thread commit/rollback pollution); 5 admin managers pooled per-KB on app.state and closed at shutdown; PermissionManager uses open_sqlite (WAL + busy_timeout); StoreBackendFactory raises on unknown backend type instead of silently falling back to local; RemoteBackend.count/clear raise instead of masking a misconfigured empty KB.
• P3 perf — delete_by_doc uses a doc_path→int_ids KV index (was O(n) full vector scan per delete); BM25 _df maintained incrementally (was full Counter rebuild per op); LocalBackend.search skips rows missing _distance with a warning (was treating missing as best match); get_document_by_metadata uses json_extract (was O(n) full-table scan); AuditLogger prunes on construction + streams exports via keyset pagination; version_manager caches record counts (60s TTL) instead of a full COUNT per snapshot.
• Deadlock fix — SqliteBase._db_lock switched Lock → RLock to fix a re-entrant deadlock in EmbeddingCache (locks-then-calls-_get_conn, which also locks during lazy connection create). 243 tests green, ruff clean.

What's New — Remote Backend (full-stack)

• Remote vector store backend (#remote-store) — RemoteBackend (store/remote_backend.py) is now a real httpx client that delegates vector add/search/keyword/delete/count/clear to a remote fusion-rag node over HTTP, instead of the old NotImplementedError stub. The matching server half is a new /kb/bases/{kb_id}/store/* surface (api/routes_store.py, auth via X-API-Key), so one fusion-rag instance can act as the vector store for another. Opt in with FUSION_RAG_STORE_BACKEND=remote + FUSION_RAG_REMOTE_ENDPOINT=http://node-b:11436; default stays local (zero behavior change). Metadata stays local (split-store model: the remote owns vectors/BM25, the caller owns document metadata/audit/trajectory). The /store/* path is exempt from the gateway-origin gate (tenant.py) since it is authenticated M2M, not a gateway-routed user request. 10 new tests (round-trip via in-process ASGI transport + config + server auth). Also fixes a pre-existing LocalBackend.delete_by_doc count bug (LanceDB DeleteResult.num_deleted_rows was never read, so deletes always reported 0).
• Authoritative tenant resolution via fusion-identity (#68) — retires the forgeable X-Fusion-Tenant blind-trust from #61. When FUSION_RAG_REQUIRE_IDENTITY=1, /kb/* requests require Authorization: Bearer <JWT>; fusion-rag verifies the JWT via fusion-identity's POST /api/v1/auth/verify (service-token-gated, revocation + tenant-status enforced by the identity service) and takes the authoritative tenant from the JWT tid claim. The resolved tenant feeds the same _request_tenant contextvar, so KB scoping (#61) and per-tenant collection isolation (#66) keep working unchanged. X-Fusion-Tenant is demoted to defense-in-depth (must match the JWT tid if present, else 401). No/invalid/revoked JWT → 401; unreachable identity → fail-closed deny. fusion-identity is an HTTP service dependency (port 11470), not a Python import — fusion-rag talks to it over plain HTTP. Default off; single-tenant local-first dev unaffected. 12 tests (forged header 401, invalid JWT 401, valid JWT scoped, cross-tenant 404, header/JWT mismatch 401, revoked 401, exempt paths, integration-off regression guard, client unit).
• Qdrant vector backend + per-tenant collection isolation (#66) — optional QdrantBackend (store/qdrant_backend.py) sinks vector storage to Qdrant and picks the collection PER REQUEST from the authoritative tenant (X-Fusion-Tenant): a tenant-A upsert lands in tenant_id_A, a tenant-B search queries tenant_id_B — cross-tenant vectors are physically invisible (collection-per-tenant is the data-layer isolation tier; the KB-level tenant field from #61 is the logical tier). When no tenant is in effect (single-tenant dev, isolation off), a shared fusion_rag_kb_{kb_id} collection is used so the default path still works. chunk_id (str) maps to a Qdrant point id (int) via a deterministic blake2b hash, so re-ingest overwrites cleanly. Keyword search stays in-process (BM25), exactly like the other backends — Qdrant is a vector primitive. Opt in with FUSION_RAG_STORE_BACKEND=qdrant + QDRANT_URL (:memory: = in-process local mode, no server needed; otherwise the remote Qdrant URL); pip install 'fusion-rag[qdrant]'. Default stays local (LanceDB), so no migration. 12 tests (round-trip recall, BM25, delete, cross-tenant isolation acceptance, default-collection, dim-mismatch) run with :memory: — no Qdrant server needed in CI.

What's New in v0.7.0

• fusion-store HNSW vector backend (#47) — optional FusionStoreBackend sinks vector storage/retrieval to the in-tree fusion-store HNSW index via its PyO3 binding. Opt in with FUSION_RAG_STORE_BACKEND=fusion-store; default stays local (LanceDB), so no migration. fusion_store uses int ids while fusion-rag uses str chunk ids — the backend maintains a str↔int mapping in fusion-store KV and persists an id counter in a sidecar. Chunk metadata rides in fusion-store KV (put_kv); BM25 keyword search stays in-process (fusion-store is a vector primitive only, per its PRD boundary). Hybrid search (vector + BM25 + RRF) keeps working. 9 backend tests (skip on CI where the binding isn't installed).

What's New in v0.6.7

• fusion-core HTTP client migration (PR #41) — LLM modules (reranker, contextualizer, query_rewriter, rag_chain, graph_rag, evaluator, streaming, routes._generate_answer) now use fusion_core.http_client.get_async_client (shared connection pool, LRU-keyed by loop+base_url) + with_retry (auto-retry on 429/5xx + transient errors). Auth headers passed per-request, not baked into pooled client. SSE streaming and embed/client keep raw httpx.
• D-H3 empty-content guards — success-path empty/whitespace LLM responses now degrade explicitly (logged error/fallback) instead of returning empty as a valid result.
• CLAUDE.md re-checked in — un-ignored and refreshed: accurate architecture tree (parse/, permissions/, store split, 7 route files, 7 new engine modules), 170 tests green, 5 missing env vars documented, fusion_core dependency + ruff lint documented.

What's New in v0.6.6

• RRF Hybrid Search Fix — _rrf_fusion no longer applies a cosine-scaled similarity_threshold (0.3) to rank-fusion scores (max ~0.066); hybrid RRF search now returns matches instead of silently empty results. Regression test added.

What's New in v0.6.0

• Version Snapshots — Hard-link based KB snapshots with rollback
• Incremental Sync — MD5 hash + mtime change detection for directories
• AST Chunking — Python AST-aware code chunking (auto-detect)
• Search Templates — 3 presets (general/code/design) + custom templates
• Audit Logging — Full search audit trail with JSON/CSV export
• Fine-grained Permissions — Role-based ACL with path-prefix inheritance
• StoreBackend Abstraction — LocalBackend (LanceDB) + RemoteBackend (extensible)
• Bench API — Search latency benchmarking with SQLite results
• Cloud Embed Fallback — Auto-fallback to cloud API when local fails
• Git Repo Indexing — Clone + .gitignore-aware file indexing
• Routes Split — Monolithic routes.py → 4 focused sub-modules

---

License

Apache License 2.0 — see LICENSE for details.

Acknowledgments

• fusion-mlx — Apple Silicon model serving
• LanceDB — Vector database
• LightRAG — Reference architecture