mcp-daizhigev20

MCP Server for searching and processing ancient Chinese TXT texts from the daizhigev20 GitHub repository.

Overview

mcp-daizhigev20 is a Python-based MCP (Model Context Protocol) server that provides:

• Index Building: Enumerate TXT files from a GitHub repository and build a local SQLite index with OpenCC simplified/traditional Chinese conversion
• Book Search: Multi-layer matching (exact, normalized, prefix, fuzzy) with simplified/traditional Chinese compatibility
• Document Inspection: Comprehensive structural analysis including encoding detection, character analysis, heading detection, and bracket analysis
• Format Conversion: Convert ancient Chinese texts to structured JSON or RAG-ready JSONL format

Installation

pip install mcp-daizhigev20

Or via uvx:

uvx mcp-daizhigev20

Requirements

• Python >= 3.12
• OpenCC data (included via opencc-python-reimplemented)

Quick Start

1. Build the index (required first step):

MCP tool: build_index

This enumerates all TXT files in the daizhigev20 repository and builds a local SQLite index.

2. Search for a book:

MCP tool: search_books(query="资治通鉴")

3. Select a book from search results:

MCP tool: select_book(search_id="...", rank=1)

4. Inspect the document structure:

MCP tool: inspect_book(book_id="...")

5. Convert to your desired format:

MCP tool: convert_book(book_id="...", output_format="jsonl")

Tools

Configuration

Environment Variables

Automated Index Updates

Use external schedulers to keep the index up to date:

cron (every 6 hours):

0 */6 * * * daizhige-mcp-index

systemd timer: Create a service that calls build_index via MCP client.

Note: scheduled index updates are not part of the 0.1.0 server itself — use cron, systemd timer, or CI pipelines.

Output Formats

Structured JSON (output_format="json")

Preserves volumes, chapters, sections, paragraphs, and annotations with full source metadata.

{
  "source": { "repository": "...", "git_sha": "..." },
  "metadata": { "title_original": "..." },
  "sections": [...],
  "paragraphs": [...]
}

RAG JSONL (output_format="jsonl")

Structure-aware chunking with configurable chunk_size (default 1200) and chunk_overlap (default 150). Chunks respect section boundaries and use paragraph/sentence-aware splitting.

{"id": "chunk_000001", "text": "...", "metadata": {...}}
{"id": "chunk_000002", "text": "...", "metadata": {...}}

Chunking Strategy

• Priority 1: Respect volume/chapter/section boundaries
• Priority 2: Split at paragraph boundaries
• Priority 3: Split at sentence punctuation (。！？；)
• Fallback: Safe window (400 chars) for unpunctuated classical text

Future Plans (not in 0.1.0)

• TEI/XML output format
• Markdown output
• EPUB output
• IIIF Annotation export
• CSV/database import format

License

MIT