doclens

Structure-aware document retrieval — FTS5/BM25 keyword search over document trees, with an interactive TUI and a PWA Web UI.

doclens parses documents into tree structures (headings, classes, functions…) and searches them with FTS5/BM25 keyword matching — no embeddings, no chunking, no vector DB required. Works entirely offline.

Features

Installation

pip install doclens

Requires Python ≥ 3.10.

Quick setup:

# Index your documents
doclens index --force

# Search from CLI
doclens search "authentication"

# Or launch the Web UI (opens browser automatically)
doclens gui

CLI Reference

doclens <command> [--workdir DIR]

Quick Start

1. Index your documents

# Index the current directory
doclens index --force

# Or specify a working directory
doclens index --workdir /path/to/project

doclens automatically discovers supported files (.md, .py, .pdf, .docx, .xlsx, …) and skips common ignore patterns (.git, node_modules, __pycache__, .venv).

2. Search

doclens search "authentication flow"
doclens search "量子 计算"          # Chinese supported via jieba

# Structured query
doclens search_v2 '{"type": "and", "terms": ["auth", "token"]}'

3. Interactive TUI

doclens

Opens the full terminal UI with live preview, command history, and keyboard navigation.

4. Web UI

doclens gui
# INFO: Uvicorn running on http://127.0.0.1:7860

Browser opens automatically. Port may vary if 7860 is in use — check the startup log.

5. Ask the AI

doclens ai "How does the authentication system work?"

doclens first retrieves relevant document sections, then sends them to Anthropic Claude as context for a grounded answer.

Claude Code Integration (MCP KB Q&A)

doclens auto-starts an MCP server (Streamable HTTP) inside both the TUI and the GUI, exposing the knowledge base as two MCP tools — search_kb and read_document. Any MCP-compatible client (Claude Code, Cursor, Cline, …) can connect and answer questions grounded in your indexed documents, with zero embedding / vector DB.

How it works

• The MCP HTTP server runs in a background thread in-process, sharing the same IndexManager as the TUI/GUI — so live reindexing via the file watcher applies to MCP queries too.
• It listens at http://<host>:<port>/mcp. The URL is printed in the startup log on every launch.
• Loopback (127.0.0.1) by default, no auth. Binding to a non-loopback address requires a bearer token (CORTEX_MCP_TOKEN) — the server refuses to start otherwise.

Setup

1. Start doclens (this also starts the MCP server):

doclens gui          # Web UI mode
# or
doclens              # TUI mode

Read the MCP URL from the startup log:

MCP server: http://127.0.0.1:7880/mcp

2. Register it in Claude Code (once per project):

claude mcp add --transport http doclens http://127.0.0.1:7880/mcp --scope local
claude mcp list      # expect: doclens: ... ✔ Connected

Scopes: local (default — this project + you, not committed), user (global), project (.mcp.json, committed).

3. Restart your Claude Code session if it was already running — MCP servers load only at session start.

4. Ask. In Claude Code, ask anything about your indexed docs; it will call search_kb / read_document automatically. For a focused, KB-only answer, restrict the session to the two tools:

claude -p "量子密钥分发 QKD 的基本原理是什么？" \
  --allowedTools "mcp__doclens__search_kb" "mcp__doclens__read_document"

Bundled skill: kb-ask

doclens ships a Claude Code skill (source: doclens/claude_code_skills/kb-ask/skill.md) that codifies the full KB Q&A workflow: MCP-connected prerequisite check, FTS multi-query strategy, read_document deep-read, source-citation rules, and a no-fabrication constraint.

On TUI/GUI startup, doclens checks ~/.claude/skills/kb-ask/ and, if the skill is missing or out of date, prompts to install/overwrite it (skipped silently in non-interactive terminals). After the first install, restart your Claude Code session and invoke it anywhere:

/kb-ask 新能源汽车技术有哪些

Configuration

MCP behavior is controlled by these env vars (same .env as the rest of doclens):

Troubleshooting

Configuration

doclens reads .env in the project root. Copy and customize:

cp doclens/.env.example .env

Key variables:

Architecture

┌─────────────────────────────────────────────┐
│                  TUI (Textual)              │
│  ┌───────────────────────────────────────┐  │
│  │  HeaderBar │ ContentArea │ InputBox   │  │
│  └───────────────────────────────────────┘  │
└────────────────────┬────────────────────────┘
                     │
┌────────────────────▼────────────────────────┐
│           Web UI (Lit + Shoelace PWA)      │
│         FastAPI + SSE streaming             │
└────────────────────┬────────────────────────┘
                     │
┌────────────────────▼────────────────────────┐
│         IndexManager + Scoring              │
│    TreeSearch (FTS5 + BM25)                │
└────────────────────┬────────────────────────┘
                     │
┌────────────────────▼────────────────────────┐
│    treesearch/  —  parsers, indexer, FTS5  │
│    planify/     —  AI agent runner          │
└─────────────────────────────────────────────┘

• treesearch: Powers the indexing and retrieval engine (FTS5/BM25 over document trees)
• planify: Drives the AI agent, session management, and tool execution
• doclens: Ties them together — CLI, TUI, Web UI, event bus, and file watcher

License

Apache License 2.0 — see LICENSE.