PagePilot

Standalone, provider-agnostic PDF page-indexing SDK. Layout heuristics build a nested tree index of a document; retrieval and citation-answer chat run on an injected LLM callable — PagePilot never imports a provider SDK, so it works with any model/tool (OpenAI, Gemini, Groq, Copilot, LiteLLM, …).

Vectorless, reasoning-based retrieval: the LLM navigates a table-of-contents tree the way a human reader would, then answers with exact page (and document) citations. No vector DB, no chunking required — but chunks with section paths are a first-class export for hybrid/vector pipelines.

Install

pip install pagepilot          # from PyPI (0.3.0+)
# or, for development:
python -m venv .venv && .\.venv\Scripts\activate
pip install -e ".[dev]"

Quick start

from pagepilot import PagePilotClient
from pagepilot.llm import ChatCompletion

def llm(messages, *, model=None, max_tokens=None):
    return my_provider.chat(messages, model=model, max_tokens=max_tokens)

client = PagePilotClient(ChatCompletion(llm, chat_model="your/model"),
                         summary_concurrency=1)   # gentle by default

entry = client.submit_document("report.pdf")       # llm-free structure, or:
# entry = client.submit_document("report.pdf", mode="skip-summaries")  # never calls the LLM
doc_id = entry["doc"]["id"]

# OpenAI-shaped answer dict + cited result
answer = client.chat("What was the 2023 margin?", doc_id=doc_id)
print(answer["choices"][0]["message"]["content"])

# Typed Answer with page + document sources
typed = client.answer([{"role": "user", "content": "…"}], doc_id=doc_id)
print(typed.content, [c for c in typed.citations])

Multi-turn conversations are context-aware: page and document selection receive the earlier turns, so follow-ups ("…and the second part?") resolve correctly.

Multi-document chat

ids = [client.submit_document(d)["doc"]["id"] for d in ("a.pdf", "b.pdf")]
answer = client.chat("Compare the risk factors across these filings.",
                     doc_id=ids)
print([f"{s.doc_name} p.{s.page}" for s in client.answer([], doc_id=ids).sources])

The client picks relevant documents by reasoning over the library index, then retrieves pages per document and cites the correct source document.

Change detection

r1 = client.submit_document("report.pdf")   # {"status": "ingested"}
r2 = client.submit_document("report.pdf")   # {"status": "unchanged"} (same sha256)
r3 = client.submit_document("report.pdf", force=True)   # re-index

Chunks & export (for vector pipelines)

from pagepilot import chunk_document

chunks = chunk_document(client.raw_tree(doc_id), source=doc_id)
for c in chunks[:3]:
    print(c.section_path, c.page, c.content[:60])

print(client.export(doc_id, fmt="json"))       # json | csv | markdown | tree

Index health (zero LLM calls)

from pagepilot import assess_library
for r in assess_library(client.storage_path):
    print(r["name"], r["coverage"], r["problems"])

Or via CLI: python scripts/assess_index.py --storage <store> (exit code 1 on failures — CI-safe).

What it does

1. Detect — per-page text-layer analysis (pagepilot.detect)
2. OCR — optional, via OCRmyPDF + Tesseract for scanned PDFs (pagepilot.ocr)
3. Layout extract — pure heuristics (no LLM): columns, body-font stats, header/footer stripping, TOC recognition, heading candidates, doc title, embedded-PDF-bookmark merge, graceful degradation to coarse page groups (pagepilot.layout)
4. Tree — node walk / validation / pruning; typed TreeNode view (pagepilot.tree, pagepilot.models)
5. Summaries — bottom-up, gentle concurrency (pagepilot.summarize)
6. Library — corpus-level document selection for multi-doc chat (pagepilot.library)
7. Retrieval — one-shot page pick from the tree, history-aware (pagepilot.retrieve)
8. Chat — cited answers (page + document), OpenAI-shaped output, streaming surface (pagepilot.chat)
9. Chunk/export — hierarchy-aware chunks + JSON / CSV / Markdown / tree serializers (pagepilot.chunk, pagepilot.export)
10. Assess — zero-LLM index health reports (pagepilot.assess)
11. Storage — JSON per-doc dirs under a user storage root (pagepilot.storage)

Examples

python examples/quickstart.py path/to/report.pdf other.pdf
python examples/export_chunks.py report.pdf out/
python examples/index_health.py path/to/report.pdf
python -m pagepilot.demo path/to.pdf     # deterministic fake-LLM demo

Examples run against an injected fake LLM — no keys, no network.

Design notes

• No provider coupling — the LLM seam (llm.py) is the only network-ward interface; providers own their retry/quota/header logic.
• Cheap ingest — structure extraction is entirely heuristic; summaries are the only per-ingest LLM cost, or zero with mode="skip-summaries".
• Honest degradation — documents that defeat heuristics fall back to page groups, never fabricated subsection structure.
• Traceable answers — every statement carries [document p.page] citation from the actually-read pages.

Development

python -m venv .venv
.\.venv\Scripts\activate        # Windows PowerShell
python -m pip install -e ".[dev]"
python -m pytest -q

License

MIT — see LICENSE.