htpd-reader 0.1.2.post1

What it is  ·  Download  ·  Quick start  ·  Use with your AI  ·  Format  ·  Results  ·  Roadmap

_{Real commands on the NXP i.MX 93 data sheet (answer output shortened).}

What is HTPD?

You have a 300-page datasheet or a 5000-page reference manual and a question about it. You upload the PDF to an AI, and it answers slowly, misses details, mixes up tables, cannot tell you the page - or invents an answer that is not in the document at all.

HTPD fixes the input instead of the model. It has two parts:

flowchart LR
    PDF["📄 PDF<br/>56 MB, 5652 pages"] -->|htpd convert| H["HTPD<br/>sections · pages · tree"]
    H -->|upload| CHAT["AI chat<br/>(file + skill)"]
    H -->|htpd-read / MCP| AGENT["AI agent<br/>(Claude Code, Antigravity, Codex)"]
    CHAT --> ANS["Answer from the file<br/>with page numbers"]
    AGENT --> ANS

Quick start

Download (no build needed)

Get the files from the release page:

tar xzf htpd-0.1.2-linux-x64.tar.gz
cd htpd-0.1.2-linux-x64
./htpd convert manual.pdf --single           # writes manual.htpd - give it to your AI

Build from source

Requirements: Linux (tested on Ubuntu 24.04, also in WSL), g++ 13+, make or CMake, Python 3.10+.

git clone https://github.com/narendrachampaneri/htpd.git
cd htpd

./scripts/get_pdfium.sh        # downloads the PDF engine once (PDFium, about 4 MB)
make                           # builds build/htpd
make test                      # optional: the C++ tests

Convert a PDF

./build/htpd convert manual.pdf --single     # one file: manual.htpd  (for AI chats)
./build/htpd convert manual.pdf              # a folder: manual.htpd/ (for tools and agents)

Ask a question from the command line

python3 -m pip install -e .                  # installs htpd-read and htpd-mcp

# try it on the sample file that comes with the repository
htpd-read ask examples/single-file/sample.htpd "What is the maximum bus voltage?"

[page 2, 2.1.1 Limits]
  Parameter | Min | Typ | Max | Unit
  Bus voltage | 200 | 400 | 480 | V
...

Each answer line is copied word for word from the file, with its page and section; --context adds the whole sections for the AI to read.

htpd-read ask examples/single-file/sample.htpd "Which OCPP version is supported?"

Not found in this file. (the file never mentions OCPP)

Then use your own file the same way: htpd-read ask manual.htpd "your question".

Use it with your AI

1. Any AI chat - just upload the file

htpd convert manual.pdf --single writes one .htpd file. It begins with the reading rules for the model - answer only from this file, give the page of every fact, say "Not found in this file." when the answer is not there - followed by the contents and every section with its pages.

Upload it to Claude, ChatGPT or any other assistant instead of the PDF. If an app does not accept the extension, rename it to manual.htpd.txt - it is plain text.

Best for documents that fit in the chat's context (a datasheet of a few hundred pages). For very large manuals, use the skill below.

For a ChatGPT custom GPT or a Gemini Gem that always has your document and follows the rules, see docs/AI-APPS.md (instructions ready to paste).

2. Apps with skills - the htpd skill

Download htpd-skill.zip from the release page, or build it from the repository:

python3 scripts/build_skill.py               # writes dist/htpd-skill.zip

The skill runs a small script that needs only Python - no packages, no model. It searches the file, returns the exact lines with [page, section], and says why when something is not in the file. Because the model reads only what the script returns, this also works for manuals far too large for any chat.

3. Agents with MCP - the htpd-mcp server

pip install htpd-reader        # the reader: htpd-read and htpd-mcp, nothing else needed

The server is also listed in the official MCP registry as io.github.narendrachampaneri/htpd (MCP clients start it with uvx htpd-reader).

claude mcp add htpd -- htpd-mcp --bundles ~/manuals        # Claude Code

For Antigravity, Codex or any MCP client: command htpd-mcp, arguments --bundles /path/to/your/htpd/files. Tools: list_bundles, contents, open_section, search, ask, figure (the picture of a figure, as an image), check. Standard library only.

4. Claude Code - the htpd plugin

One install gives Claude Code everything: the skill that answers from .htpd files, a skill that converts PDFs, and the MCP server.

/plugin marketplace add narendrachampaneri/htpd
/plugin install htpd@htpd

Then ask in plain words - "convert datasheet.pdf to htpd", "what is the maximum NPU frequency in datasheet.htpd?" - or run /htpd:convert datasheet.pdf --single. The first conversion downloads the converter of the release (Linux x64 and WSL, about 4 MB, checked by SHA-256) once; an htpd already on the PATH is used instead. Only Python 3.10+ is needed; the PDF never leaves the computer.

The format

An HTPD document is plain UTF-8 text: Markdown with a few fields. Any model, editor or grep can read it.

<!-- htpd section {"n":14,"id":"sections/0014-4-1-5-maximum-frequency-of-main-modules","number":"4.1.5","level":3,"pages":[18,18,18],"parent":"sections/0008-4-1-chip-level-conditions"} -->
## [S14] 4.1.5 Maximum frequency of main modules

Path: 4 Electrical characteristics > 4.1 Chip-level conditions > 4.1.5 Maximum frequency of main modules - page 18

<!-- page 18 -->

```text
Cortex®-A55 cores | 0.9 GHz | 1.4 GHz | 1.7 GHz
DRAM | 933 MHz | 1400 MHz | 1866 MHz
NPU | 500 MHz | 800 MHz | 1000 MHz
```

• Sections follow the document's own headings (bookmarks, numbered headings, font sizes) and form a tree: parent, children, previous, next.
• Page marks <!-- page N --> stand above the text of every page.
• Tables are rows with | between cells; code and listings keep their spacing.
• Two forms, same content: a single .htpd file (for chats) and a bundle folder with one file per section (for tools), which is also a valid Open Knowledge Format bundle.

The full specification - every field and every rule - is in docs/FORMAT.md. Examples: examples/single-file/sample.htpd and examples/sample.htpd/.

Results

The converter on real documents (single file, figures and OCR on, a 2-core cloud machine):

Most of a file with figures is their pictures, which stand at its very end: the text comes first. --figures off leaves them out (the i.MX 93 Reference Manual is then 14 MB). Without the Tesseract program the converter skips OCR and says which pages it could not read.

The reader on 133 test questions about two public manuals (eval/report.md, written by the evaluation harness):

The text handed to the AI is about 6 % of the whole document.

The converter on 45 PDFs of known text, made by Word, LaTeX, Chrome, wkhtmltopdf and LibreOffice slides, scanned, and with letters drawn as shapes (tests/zoo): 99 % of the text of the English ones in order, 96 % of their headings, 89 % of their table rows exact; Hindi and Gujarati from Word 91 % and 99 % of the text.

Robustness. A stress test makes 34 difficult PDFs locally - two columns, sideways tables, rotated pages, 1500 pages, huge tables, CJK and Unicode text, encrypted, truncated and corrupted files, a scanned page, LaTeX and Word exports - and checks that the converter never crashes or hangs, that every bundle is valid, and that no word is lost.

Tests. 96 C++ tests and 150 Python tests run on every change, the Python tests also without PyYAML, as the skill does in a chat sandbox.

Roadmap

HTPD is version 0.1. It works today; these are the known next steps.

• Converter: sections, pages, tables, code, headers/footers, sideways pages, damaged files
• Structure for documents without headings (parts of a few pages) and a map of the document
• Figures stored as pictures with caption and page; OCR of scanned pages and text in pictures (Tesseract, optional)
• Single-file .htpd with reading rules for AI models
• Reader, "not found" rule, ask, MCP server, skill for AI apps; figures handed to the AI as images
• A small trained re-ranker (no GPU, a few kilobytes) - the right section first for 78 % of test questions instead of 66 % without it (model/README.md)
• Other document types: Word, HTML, plain text, images
• "Not found" for questions in the user's own words (a word the document writes differently)
• Windows build

Known limits: PDF only; what a chart or diagram means is only in its picture (an AI must look at it); formulas lose their layout; scanned pages need the free Tesseract program; tables are rows of text (a table whose first column wraps can come apart); tested mainly with English technical documents. Details: docs/CONVERTER.md.

Project layout

include/ src/ cli/      the converter (C++23, PDFium is used in one file only)
reader/htpd_reader/     the reader: bundle, check, search, retriever, re-ranker, packer, answer, MCP server
skills/htpd/            the skill for AI apps (scripts/build_skill.py packs it)
skills/convert/         the plugin's skill that converts PDFs
.claude-plugin/         the Claude Code plugin and its marketplace (this repository is both)
eval/                   evaluation harness, baselines, report
model/                  the training of the re-ranker
tests/                  C++ tests, test PDFs, the stress test
docs/                   FORMAT.md · CONVERTER.md · READER.md · BACKGROUND.md
examples/               a converted document in both forms

Contributing

Issues and pull requests are welcome. Before you change anything, read AGENTS.md; the short version:

• All tests pass before a change is merged: make test and python3 -m pytest.
• Every new rule in the code comes with a test that fails without it.
• Never lose text and never make text up.
• A change to what is written into an HTPD file updates docs/FORMAT.md in the same commit.

License

HTPD is open source under the Apache License 2.0: you may use, change and share it, also in commercial products, as long as you keep the copyright and license notices (NOTICE) and mark what you changed. The license includes a patent grant from every contributor.

PDFium, which the converter uses, is not part of this repository and has its own license (BSD-style with an Apache-2.0 part); after scripts/get_pdfium.sh its license texts are in third_party/pdfium/licenses/. If you give the program to others together with libpdfium.so, give them those files too.