TreeSearch

简体中文

Structure-aware document retrieval via tree-structured indexing.

No vector embeddings. No chunk splitting. TreeSearch parses each file into a heading tree that mirrors the document's own structure, then builds an FTS5 keyword index (SQLite's built-in full-text search) over the tree nodes. Every hit comes back with its chain of ancestor headings, so you always know which section it came from — a natural fit for keyword search over code bases, technical documentation, and note collections.

Why TreeSearch

Vector RAG pipelines chunk documents, embed the chunks, and return approximate neighbors that are hard to explain or reproduce. TreeSearch keeps the structure intact and searches it with classic full-text ranking instead:

• Explainable — every hit is a tree node with a title, its ancestor headings, and source line numbers.
• Reproducible — FTS5 + BM25 ranking is deterministic: the same query over the same index always returns the same results.
• Structure-aware — a hit under Configuration → Authentication is distinguishable from a passing mention elsewhere.
• Fully local — one SQLite file holds everything: document trees, the FTS5 index, and incremental metadata. No embedding service, no vector database, no network calls.

Installation

Requires Python ≥ 3.10. The PyPI distribution name is treesearchlib; the import name is treesearch.

pip install treesearchlib            # light core: Markdown / code / JSON / CSV / plain text
pip install "treesearchlib[all]"     # all capability groups except `legacy` (see table)

Install capability groups on demand:

Missing groups degrade gracefully: affected formats fall back to plain-text indexing and tokenization falls back — TreeSearch never crashes over a missing optional dependency. Outlook PST parsing is built in and ships a pst-extract sidecar binary with the package, so it works out of the box.

Watch mode (treesearch watch) needs the watchdog package, which is not part of any extras group — install it separately:

pip install treesearchlib watchdog

Quick Start

Library usage (most common)

from treesearch import TreeSearch

# Lazy indexing — the index builds itself on the first search
ts = TreeSearch("./docs/")
results = ts.search("How to configure voice calls?")

Useful variations:

# Eager indexing with explicit glob patterns
ts = TreeSearch()
ts.index("docs/*.md", "src/*.py")
results = ts.search("auth flow")

# In-memory mode — no .db file on disk; indexes vanish when the process exits
ts = TreeSearch("docs/", db_path=None)
results = ts.search("voice calls")

# Persist / reload an index file
ts.save_index("./my_index.db")
ts.load_index("./my_index.db")

# Batch: many queries share one warm index, results in input order
results = ts.batch_search(["authentication flow", "database schema", "rate limits"])

CLI

# Default mode: lazy index + search in one shot
treesearch "How does auth work?" src/ docs/

# Build a persistent index, then search it repeatedly
treesearch index --paths src/ docs/
treesearch search --query "FTS5 search"

# Keep the index up to date automatically (needs: pip install watchdog)
treesearch watch --paths docs/

Both treesearch and python -m treesearch invoke the same CLI.

CLI Reference

Default mode (no subcommand) — lazy index + search:

Subcommands:

Supported Formats

Extensions without a registered parser fall back to plain-text indexing.

Library API

TreeSearch is the only class most users need:

The package also exports lower-level building blocks for advanced use: build_index, md_to_tree, text_to_tree, search / search_sync, TreeSearcher / PathResult / build_query_plan (the Best-First tree search), Document with tree serialization helpers, FTS5Index, and the config trio TreeSearchConfig / get_config / set_config.

Configuration

Priority from high to low: set_config(TreeSearchConfig(...)) → TREESEARCH_* environment variables → built-in defaults.

Environment variables:

Frequently tuned fields on TreeSearchConfig itself: top_k_docs (3), max_nodes_per_doc (5), search_mode (auto), and the FTS column weights (fts_title_weight 5.0, fts_summary_weight 2.0, fts_body_weight 10.0).

Design Notes

• Tree-structured index — every document becomes a heading tree whose nodes carry titles, summaries, and source line ranges. FTS5 is built at node level, so a hit is traceable to an exact section.
• Zero vectors, zero chunking — BM25 keyword ranking over tree nodes; results are explainable and reproducible.
• Two search modes — tree mode runs a Best-First Search over each document tree starting from anchor hits; flat mode returns raw FTS5 matches. auto degrades to flat for code-only corpora.
• Incremental indexing — stat or content fingerprints detect changes per file; orphaned entries are pruned on directory walks; the schema version is folded into every fingerprint, so upgrading TreeSearch transparently triggers a rebuild of stale indexes.
• Chunked indexing — large corpora are processed in chunks of 500 files (parse → commit → release memory), keeping memory flat all the way to millions of documents.
• Shadow Markdown — binary formats (PDF, DOCX, …) get hidden .md shadows so the ripgrep fallback path can still search them.

License

Apache-2.0. Repository and issue tracker: https://github.com/zhlhaohao/doclens