pagebound

Shared PDF ingestion cache. Converts a PDF once, into structured markdown and JSON that carry page and bounding-box provenance for every block, and serves that result to every tool that needs it, and to every agent: the CLI is written so that an LLM agent can find a paper, open its full text and cite a passage by page from --help alone.

Why

Converting a research PDF into something a machine can reason about is expensive: the Docling model set is one to two gigabytes, and a warm conversion of a fifteen-page paper takes fifteen to thirty seconds. A cost that high, paid repeatedly and independently by every tool on the same machine, is the classic case for a shared cache.

Every tool that reads papers ends up solving a slice of the same problem: how to invoke the converter, where to cache the result, how to keep images out of the text, how to carry page and bounding-box provenance, what to do when the converter crashes, how to skip a scan that has no text. pagebound is the union of those slices, written once.

What it is not

Not a converter. Docling does the conversion, and does it well. pagebound invokes it as an external tool, so the heavy pipeline never enters any consumer's dependency tree.

Not a Zotero plugin. A Zotero library is one way to find PDFs; a directory is another. Both are source adapters.

What is actually scarce

Not the conversion. The contract: a cached artifact that records what produced it, so a consumer can tell whether it still matches the PDF on disk. Existing tools in this space cache by filename, which fails silently the moment a PDF is replaced.

pagebound keys every artifact on the source hash and the recipe that produced it: converter, resolved version, and options. A Docling upgrade produces a new entry rather than silently changing the meaning of an old one.

For agents and LLM tools

The most common reader of a cached paper is not a program but an agent in a conversation: "what does Bandara et al. 2015 say about rigour?", "is this quote really in the paper?", "brief me on this before I read it". pagebound is shaped for that reader.

• A handle you can say. Every paper has a twelve-character sha256 prefix, such as 6cbb126970f0. It names the object directory, needs no state beyond the hash, and stops resolving exactly when the PDF behind it is replaced. pagebound path 6cbb126970f0 prints the directory; pagebound path BAK9F4R4 (a Zotero item key) and pagebound path 10.1108/eb024320 (a DOI) do the same.
• Find, then read. pagebound list --json is the whole index as JSON: title, authors, year, DOI, sha256, and whether the text is cached. An agent greps it for a surname and a year, takes the sha256, and reads document.md in the directory pagebound path prints. Reading never runs Docling, so it costs milliseconds and works on a machine without it.
• Cite by page. document.json carries every block with its page and bounding box, and md_start/md_end offsets into document.md, so a passage found in the markdown maps back to "page 4, this box" for a citation or a highlight, with no fuzzy matching.
• Help that stands alone. Every command's --help states what it prints, what it needs running, its exit codes (0 found, 1 not in the cache or gave up, 2 usage error) and the fields of any JSON output, so an agent can drive the CLI without reading this file.
• Honest text. The markdown carries the source characters, never & or \_, so a verbatim quote check is a string search. Text that came out of OCR is marked as such in document.json (source.text_layer false), so an agent can weigh it.

A minimal agent skill is three commands:

pagebound list --json | jq '.[] | select(.authors[] | test("Bandara")) | select(.year == 2015)'
pagebound path <sha256 from above>
cat "$(pagebound path <sha256>)/document.md"

Installing

uv add pagebound                    # as a dependency of your project
uv add "pagebound[zotero]"          # adds pyzotero, for `pagebound sync`
uv tool install "pagebound[zotero]" # the CLI on its own, on PATH

uv is required either way: Docling is never a dependency, pagebound runs it through uv tool run, so the first conversion on a machine downloads it. Not on PyPI yet; until then, point uv at the repository.

Artifacts

Per (source hash, recipe):

Measured over 113 papers: about 1.1 MB per paper, so a 5,900-paper library costs about 6.5 GB per recipe. Every Docling upgrade is a new recipe and converts again beside the old one, so the cache grows by that much per upgrade until pagebound clean drops the superseded conversions; pagebound stats shows the cost per recipe at any time.

Two things Docling emits are not stored, and they were most of the bytes. Its JSON carries a base64 render of every page; stripping those removes 97.9% of it and costs nothing, since the result still validates as a DoclingDocument and still chunks with HybridChunker. And --image-export-mode referenced also writes a PNG of every page beside the figures; nothing references them, and on a real library they were 94% of the whole cache, about 12 MB per paper. pagebound versions before 0.1.0 stored them; pagebound clean removes them.

Objects are built in a dot-prefixed staging directory and renamed into place, so a reader sees a whole object or none; skip dot-prefixed directories when listing. document.json is the completeness sentinel: a directory without it is a tombstone or a torn write, and must not be read. tombstone.json is {"reason": ..., "tombstoned_at": ...}, the time in ISO 8601 UTC; tombstones written before the field existed have only the reason. When one source has several recipe directories, read the one whose conversion.converted_at is newest; sync always writes under the recipe that runs now, so that is the current recipe's artifact whenever one exists.

Finding a paper: the index

Artifacts are keyed on (source sha256, recipe), which is file identity, not paper identity. The bridge is index.sqlite at the cache root, filled by pagebound sync: one row per Zotero item, carrying the item's identity, the PDF's hash and location, and the bibliographic metadata Zotero curates. The row is written before the conversion is attempted, so an item whose PDF failed to convert is still findable by title or DOI, and the store says it is tombstoned rather than the index saying nothing.

items (
    library_key TEXT,  -- 'user' or 'group:<id>'
    item_key    TEXT,  -- Zotero item key
    sha256      TEXT,  -- the artifact key under objects/
    path        TEXT,  -- the PDF on disk
    size        INTEGER,
    mtime       REAL,
    title       TEXT,
    authors     TEXT,  -- JSON array of names, Zotero order
    year        INTEGER,  -- NULL when Zotero has no date
    doi         TEXT   -- lowercase, prefix-stripped; NULL when absent
)
-- PRIMARY KEY (library_key, item_key); indexes on sha256 and doi.

item_collections (
    library_key TEXT,
    item_key    TEXT,
    collection  TEXT   -- Zotero collection name, flat
)
-- PRIMARY KEY (library_key, item_key, collection); index on collection.
-- Rewritten per item on every sync that sees it, so removals in
-- Zotero propagate; like the metadata, a snapshot until the next sync.

Using it from Python

pagebound.get_document(pdf_path) returns an Artifact: document, markdown, path (the object directory, where docling.json and images/ live) and converted, true only when this call ran the converter. It resolves the current recipe, which costs one docling --version per process (about 5 s), unless you pass recipe=Recipe(...) yourself; an explicit recipe also pins that Docling version on the converter, so the provenance recorded is the Docling that ran. timeout= bounds one Docling run in seconds.

get_cached_document(pdf_path) returns the newest complete artifact across every recipe, or None, and never converts: one hash, no Docling, so it works on a machine without Docling. cache_status(pdf_path) reports cached, tombstoned (with the Tombstone's reason and time) or absent, for one recipe or, with recipe=None, across all of them, so a batch can say "12 cached, 3 to convert, 1 gave up" before paying for anything.

All three take fallback_stores=[Store(), ...]: read-only caches consulted after the writable store. A hit there is copied into store (about 1 MB per paper), a tombstone there replays, and nothing is ever written to a fallback. A tool that may write only under its own project directory points store there and lists the shared cache as a fallback.

Failures are all PageboundError. A tombstone replays as TombstonedError, a ConversionFailedError carrying reason and tombstoned_at; a tombstoned scan raises TombstonedNoTextLayerError, which is also a NoTextLayerError, so except NoTextLayerError keeps catching scans whether the verdict is fresh or recorded; a PDF that cannot be read raises SourceUnreadableError; a missing uv or Docling raises DoclingUnavailableError. force=True retries a tombstone.

A PDF with no text layer is refused and tombstoned unless ocr_scans=True (--ocr-scans on the CLI), which lets Docling OCR it under the same recipe and retries an earlier no-text-layer tombstone. Every artifact records the probe's verdict in document.source: text_layer (false means the text came out of OCR), probe_pages and probe_chars; older artifacts read them as None.

document.md is a contract: it is rendered here from the JSON's raw text, never through Docling's markdown serializer, so it carries no &, \_ or other escapes; every body block with a bounding box and non-empty text is in it verbatim, tables as rendered tables; and each such block's md_start/md_end are Python string indices into exactly that markdown, with markdown[md_start:md_end] == block.text. Running headers and footers (layer == "furniture") are in document.blocks but not in the markdown, so their offsets are None.

get_document_for_item(library_key, item_key) skips hashing when a stat shows the PDF unchanged, and get_document_for_doi(doi) reaches an item through the index, preferring the user library when a DOI is held in two; both go through the index, a snapshot of the last sync. The index itself is pagebound.index.Index (lookup, lookup_doi, lookup_sha, lookup_sha_prefix, iter_items), and pagebound.sync fills it: sync_items takes an iterable of ZoteroItem (build them with iter_items and a pyzotero client from zotero_client, or by hand), so a consumer with its own Zotero access can sync a collection or one item without the CLI. get_document(path) on its own writes nothing to the index: the sync is the only writer the index trusts.

A consumer that only reads uses pagebound.store.Store: find_sha_prefix expands a handle, artifacts lists the recipes holding a complete artifact, and latest_artifact picks the newest conversion, none of which touches Docling. Non-Python consumers open the index directly, read-only: the journal mode is WAL, PRAGMA user_version is the schema version (currently 1; any other value means delete the file and run pagebound sync), and DOIs are stored normalised, so normalise before matching (lowercase, strip https://doi.org/, http://doi.org/, https://dx.doi.org/, http://dx.doi.org/ and doi: prefixes).

Lookups are exact. Fuzzy search, and metadata for PDFs that never came through a sync, are deliberately out of scope: matching policy belongs to consumers, and the sync is the only writer the index trusts.

The CLI mirrors this: pagebound list [--json] [--collection NAME] shows what the index holds and each item's state, cached, tombstoned (tried and given up, no artifact) or absent; pagebound path REF resolves a paper to its object directory. On a terminal, list, stats and status render as tables and coloured lines for people; --json, and the paths path and convert print, are plain text and never change shape in a pipe.

REF is read by shape, so one handle works in conversation and on the command line: a sha256 prefix (lowercase hex, six or more characters, git style), a Zotero item key (eight uppercase characters, looked up in the user library), or a DOI. A prefix that matches more than one object is an error, never a guess, and one that names an object written by pagebound convert resolves even though that object has no index row. pagebound path LIBRARY ITEM still reaches items in group libraries.

pagebound path 6cbb126970f0          # sha256 prefix
pagebound path MH23L252              # Zotero item key
pagebound path 10.1016/j.jsis.2016.05.001
pagebound path group:12345 ITEM0002

The twelve-character prefix is the recommended handle: it is what the object directory is named after, it needs no state beyond the hash, and it stops resolving exactly when the PDF behind it is replaced.

path reads the cache only, so it needs neither Zotero nor Docling. When a PDF has artifacts under more than one recipe it prints the newest conversion, judged by the artifact's own converted_at; since sync always writes under the recipe that runs now, that is the current recipe's artifact whenever one exists. A paper that is indexed but has no converted text exits 1 with the tombstone's reason when there is one.

License

MIT.