brewdoc

Document to Markdown for agent systems: PDF, Word, PPTX presentations, spreadsheets, EPUB and static HTML in, one Markdown file plus a JSON receipt out. No models or OCR. Local conversion is offline and deterministic (same file, same bytes). Explicit --url acquires a page. The receipt says what was rendered, what was dropped and what the route structurally cannot carry.

Install

Choose one method. Latest release contains its complete installation guide. Python releases 0.2.0 and 0.3.0 are public. These commands select 0.3.1 and apply when that version is published; its release guide records accepted channels. Badges show published versions and main CI, not universal OS support.

pip needs an installed Python 3.12. Use explicit environment paths without activation. Linux/macOS:

python3.12 -m venv .venv
.venv/bin/python -m pip install "brewdoc==0.3.1"
.venv/bin/python -c "from importlib.metadata import version; print(version('brewdoc'))"
.venv/bin/brewdoc --self-check
.venv/bin/brewdoc file.pdf --out file.md

Windows PowerShell:

py -3.12 -m venv .venv
.\.venv\Scripts\python.exe -m pip install "brewdoc==0.3.1"
.\.venv\Scripts\python.exe -c "from importlib.metadata import version; print(version('brewdoc'))"
.\.venv\Scripts\brewdoc.exe --self-check
.\.venv\Scripts\brewdoc.exe file.pdf --out file.md

uv and pipx must already be installed. After installation, run uv tool update-shell or pipx ensurepath, open a new terminal, then run brewdoc --self-check and brewdoc file.pdf --out file.md. uv tool list or pipx list shows the managed version. Check PATH selection with command -v brewdoc on Unix or Get-Command brewdoc in PowerShell.

For an exact published version X.Y.Z, update with the environment's Python and -m pip install --upgrade "brewdoc==X.Y.Z", uv tool install --force --python 3.12 "brewdoc==X.Y.Z" or pipx install --force --python 3.12 "brewdoc==X.Y.Z". Replace X.Y.Z before running. Remove with the same Python's -m pip uninstall brewdoc, uv tool uninstall brewdoc or pipx uninstall brewdoc. pip leaves dependencies; remove the dedicated virtual environment when no longer needed. uvx uses temporary cached environments, adds no persistent PATH command and needs no tool uninstall. Choose another exact version to update.

Standalone and npm

The links and commands below work when the named release is published and its native/public consumer checks pass. Check the release guide for that version's accepted channels and targets. Download badges select assets; they do not establish runtime acceptance. The npm badge shows the registry version. Existing 0.2.0/0.3.0 releases contain Python wheel/sdist archives only.

Download SHA256SUMS beside the archive. Compare its SHA-256 before extraction with sha256sum ARCHIVE on Linux, shasum -a 256 ARCHIVE on macOS or Get-FileHash ARCHIVE -Algorithm SHA256 in PowerShell. Keep the complete brewdoc directory, including _internal and licenses. Read brewdoc/version.json for version/source/target, then run ./brewdoc/brewdoc --self-check or .\brewdoc\brewdoc.exe --self-check and use that executable with file.pdf --out file.md. Add its absolute directory to your shell profile or Windows user PATH for persistent use. Update by verifying and extracting a fresh complete release directory, self-checking it, then repointing PATH. Remove the dedicated directory and PATH entry to uninstall. Full download/extraction commands are in the generated guide attached to the published release.

Verification targets do not establish older OS support. Linux musl/ARM64 and macOS Intel are outside this initial binary set. Windows is unsigned; macOS uses an ad hoc build signature without Developer ID/notarization. Linux uses checksums without a detached publisher signature. OS policy may prevent launch.

npm requires Node 24.21.0/npm 12.2.0 for the tested flow; Python is bundled on admitted targets. Once this exact npm release is publicly verified:

npm install --global brewdoc@0.3.1
npm list --global brewdoc --depth=0
brewdoc --self-check
brewdoc file.pdf --out file.md

Temporary cached invocation:

npx --yes --package brewdoc@0.3.1 brewdoc --self-check
npx --yes --package brewdoc@0.3.1 brewdoc file.pdf --out file.md

Update with npm install --global brewdoc@X.Y.Z for an exact published version; remove with npm uninstall --global brewdoc. npx installs no persistent global tool; choose another exact version to update. Keep optional platform packages enabled. Missing packages or unsupported platforms refuse. The launcher adds no runtime downloads.

Python package installations require Python >= 3.12. Base runtime dependencies: pdfplumber (pulls pypdfium2, Pillow, cryptography) and python-calamine. HTML requires CPython 3.12-3.14 and the native Lexbor parser from this exact conditional dependency:

selectolax==0.4.13; python_version < '3.15' and platform_python_implementation == 'CPython'

Other runtimes retain the .html route and Python API but explicitly refuse HTML. On supported CPython, a missing or broken parser is an environment failure and --self-check exits 1. On an unsupported HTML runtime, self-check verifies HTML refusal and checks all other formats.

Version 0.3.0 includes URL acquisition and optional JavaScript capture. Install the extra with python -m pip install 'brewdoc[render-js]==0.3.1'. It uses QuickJS-ng 0.17.0.1 and bundled LinkeDOM 0.18.13. From this source checkout, install the locked extra with uv sync --locked --extra render-js. Run it with uv run --locked --extra render-js brewdoc --url URL --render-js. Python and standalone invocations need no Node or browser engine. npm/npx use Node as a launcher; no invocation downloads runtime components.

Usage

brewdoc file.pdf --out file.md
brewdoc slides.pptx --out slides.md
brewdoc book.epub --out book.md
brewdoc page.html --out page.md
brewdoc --url https://example.com/article --out article.md
brewdoc --url https://example.com/article --render-js --timeout 30 --out article.md
brewdoc file.xlsx                # no --out: receipt line, then the Markdown, on stdout
brewdoc file.xlsx --sheet Calc --sheet Inputs
brewdoc file.xlsm --artifact formula/sheet/000002=formulas.json
brewdoc file.xlsm --artifact vba/project/000001=vbaProject.bin
brewdoc --self-check             # renders synthetic fixtures twice, compares sha256, exit 0 when green

--sheet is repeatable. Names are exact and case-sensitive. Output follows caller order while keeping each sheet's full-workbook ordinal. --artifact is also repeatable. Use keys listed in the Markdown artifact table. Both options are workbook-only. brewdoc validates every request and output collision before writing.

URL mode accepts one public HTTP(S) HTML page with strict UTF-8 decoding. Initial page acquisition uses GET, verified TLS and default ports only. Private destinations, URL credentials and HTTPS downgrades are refused; proxy environment settings are ignored. Each redirect and resource is validated. Acquisition allows at most five redirects and an 8 MiB body limit. --timeout SECONDS sets one maximum waiting budget for loading and optional JS, default 30 seconds. It accepts positive finite seconds within the supported clock range and returns earlier when ready. Increase it for pages that need longer loading or JS waiting. Bounded cleanup and local conversion can add time after waiting ends. No login, cookie profile or bot bypass is offered. Local files and default static URL capture do not execute JavaScript.

--render-js explicitly executes the selected page's scripts locally in a separate process with bounded APIs and resources. No filesystem or shell APIs are exposed, and captures do not reuse browser state. This is not a general operating-system security sandbox. It supports DOM mutations, bounded jobs/timers and guarded GET requests. Async XHR also supports the public Algolia search endpoint used by hn.algolia.com, with its original string body and no cookies or user credentials. Other POST requests remain unsupported; search redirects are refused without replay. Supported module loading uses the native engine. Temporary localStorage starts empty for each capture; accessing it marks the result partial. Each capture starts with a writable innerWidth of 1024 CSS pixels. Width changes stay within that capture. It does not provide browser layout, iframe execution, workers, WebSockets, media or browser-wide compatibility. Native JS requires a verified QuickJS public ABI on macOS ARM64 or 64-bit glibc Linux (x86_64/aarch64). Windows, macOS Intel, musl Linux and 32-bit systems refuse JS. Missing components or unsupported runtimes refuse before fetching. See URL acquisition limits.

URL receipts use brewdoc.receipt/2 with redacted acquisition provenance and the frozen snapshot identity. Timing-only expiry preserves available valid content with acquisition.capture_status=partial and a diagnostic. The receipt distinguishes the current DOM from an explicitly marked fallback to the already acquired original HTML, where JS completion is unknown. No HTML is returned from an incomplete response. Hard network, security, heap, native fault, tamper or resource failures exit 1 and preserve an existing output file. Exit 0 confirms conversion, not script completion or article fidelity. Live acquisition can change; conversion of the same frozen snapshot remains deterministic. No 90% coverage or global JavaScript compatibility is claimed.

Combining --url with a local path or --self-check, or using --render-js without --url, exits 2 before acquisition. Invalid timeout values and an explicit --timeout without --url also exit 2. URL requests with --sheet or --artifact exit 1 with an HTML refusal receipt before HTTP.

Every successful document uses brewdoc.markdown/2. A quoted source title is followed by Metadata, Artifacts, Known omissions, Contents, then anchored content units. PDF keys are page/000001, workbook keys are sheet/000001, DOCX keys are chapter/000001, PPTX keys are slide/000001, EPUB keys follow declared spine ordinals as chapter/000001, and HTML has one chapter/000001. Metadata contains deterministic source identity, selection, content hashes, conversion tallies, and artifact capabilities. It excludes paths, timestamps, permissions, host data, and unsupported author metadata. EPUB adds source-declared publication titles, creators, languages and identifiers.

For a conversion request, one JSON receipt line goes to stdout first (wrapped here, unit_keys omitted):

{"artifacts": [], "broken_ligature_words": 0, "columns_split": 1,
 "dropped": {"cid_survivors": 0, "control_chars": 0, "ligatures": 0, "nbsp": 0,
             "non_ascii_replaced": 0, "page_numbers": 3, "pua_glyphs": 0,
             "running_heads": 0, "soft_hyphens": 0},
 "file_ok": true, "markdown_schema": "brewdoc.markdown/2",
 "not_carried": ["images, figures and the text drawn inside them",
                 "a table that spans a page break",
                 "text rotated out of the horizontal reading order"],
 "out": "file.md",
 "reason": "pdf rendered: 7 pages, 1 tables, 26 text regions, 1 column splits",
 "receipt_schema": "brewdoc.receipt/1",
 "route": "pdf", "source": "nist-sp800-145.pdf", "tables": 1,
 "text_regions": 26, "unit_kind": "page", "units": 7}

Paths resolve against the current working directory; --out creates parent directories. Exit code is non-zero on refusal; the receipt line is still printed.

Formats

.xlsx and .xlsm expose original OOXML formulas as one deterministic JSON artifact per formula-bearing selected sheet. Markdown still contains cached values. .xlsm and .xlsb can expose one complete related vbaProject.bin as opaque bytes. brewdoc does not execute macros, decompile VBA, report source-module counts, evaluate formulas, or infer dependency graphs.

HTML reads local strict UTF-8, with an optional BOM. It preserves meaningful Unicode, navigation, main content, sidebars and footers in recovered DOM order. It never fetches links or resources, executes scripts, submits forms or infers interactive values. Links use a restricted scheme policy; form actions remain inert metadata. Inline code outside tables becomes a literal fenced block; surrounding emphasis and links remain balanced. Safe links in literal code and around code/images retain destination records without changing the source literals. Table-cell code uses labelled escaped text. Quotes and list items containing standalone child blocks use explicit nesting boundaries in place of visual Markdown containers. Plain list paragraphs retain their item marker and continuation indentation. See HTML scope and limits.

EPUB reads one package with static XHTML in EPUB 2 or 3. Each declared spine item becomes a chapter, including empty and non-linear items, front matter and back matter. Publication navigation retains source hierarchy and links to generated chapter and fragment anchors. Declared metadata and meaningful Unicode are preserved. Images remain inert alt/title/target records; no pixels or external resources are read. EPUB adds no parser dependency. Malformed required content, unsafe archives, unavailable navigation targets, encrypted reading text, declared fixed layout and unsupported required media without an XHTML fallback refuse the whole book before output writes. Multiple rootfiles are outside this supported profile. See EPUB scope and limits.

The public Python API exposes EPUB, HTML and presentation rendering and generic workbook artifact access:

import brewdoc

markdown, tally = brewdoc.render_epub("book.epub")
code, receipt, markdown = brewdoc.run("book.epub", out="book.md")
markdown, tally = brewdoc.render_html("page.html")
code, receipt, markdown = brewdoc.run("page.html", out="page.md")
markdown, tally = brewdoc.render_presentation("slides.pptx")
refs = brewdoc.list_book_artifacts("book.xlsm", sheets=("Calc",))
payload = brewdoc.read_book_artifact("book.xlsm", refs[0].key, sheets=("Calc",))
code, receipt, markdown = brewdoc.run(
    "book.xlsm", out="book.md", sheets=("Calc",),
    artifact_outputs={refs[0].key: "formulas.json"},
)

artifact_outputs maps artifact keys to paths. The CLI builds it from repeated --artifact KEY=PATH.

CSV and TSV are intentionally unsupported and are not planned. LLMs can read their text tables directly, so brewdoc does not convert them to Markdown.

Planned formats, excluded ones, deferred ones and the never-list: FORMATS.md. A planned format is refused by name today. The adapter contract and the ordered checklist for adding one are in docs/architecture.md.

PDF regions, in the order tried:

What it does not do

Determinism

No timestamps or unordered iteration. Two renders of one file have the same sha256; --self-check pins it. PDF, DOCX, PPTX and workbook output remains ASCII. HTML and EPUB Markdown use UTF-8 without BOM and LF newlines, with the same bytes in the API, file output and CLI stdout. Receipt JSON keeps ASCII escaping.

Size and speed

These measurements cover the existing PDF/workbook baseline before the HTML parser addition. They do not measure HTML or EPUB conversion or the new installed footprint.

Benchmarks

Browse the corpus catalog, follow the measurement method, and inspect the full-run results. Completed runs include actual outputs and known failures; capture completion does not mean content quality passed.

Releasing

Tag-triggered, see RELEASING.md. Coding agents can start with the compact llms.txt guide. It does not imply automatic discovery by any client.

License

MIT, see LICENSE.