docmd

Convert PDFs, DOCX, and PPTX to clean, structure-preserving Markdown — from the CLI or a Python library.

Built for feeding documents into LLM and RAG pipelines, where clean Markdown beats raw text extraction.

(Published on PyPI as docmd-cli since docmd was already taken by an unrelated package — the import (import docmd) and CLI command (docmd convert ...) are unaffected.)

pip install docmd-cli
docmd convert report.pdf

from docmd import convert

markdown = convert("report.pdf")
print(markdown)

Why docmd

Great open-source document-to-Markdown converters already exist (docmd is built on Marker). The problem isn't quality — it's that running them yourself means a Python environment, several GB of RAM, ideally a GPU, and a non-trivial setup process before you convert your first file.

docmd wraps that engine with sane defaults and adds real post-processing on top (table cleanup, heading normalization) so the output is closer to what a RAG pipeline actually wants, not just raw model output.

See EVIDENCE.md for the specific real documents, real defects, and real fixes behind that claim — not a benchmark score, a log of verifiable before/after examples with the commit and regression test for each one.

A hosted API (POST a file, get Markdown back, no local setup) is planned — see ARCHITECTURE.md. It is not live yet; this repo is the open-source core, usable standalone today.

What it handles

Quickstart

CLI

pip install docmd-cli
docmd convert my-file.pdf -o output.md

Python

from docmd import convert

# From a file path
markdown = convert("my-file.pdf")

# From bytes
with open("my-file.pdf", "rb") as f:
    markdown = convert(f.read(), filename="my-file.pdf")

Installing DOCX/PPTX support

The base install (pip install docmd-cli) covers PDF only and stays lean. DOCX and PPTX need Marker's own additional dependencies:

pip install "docmd-cli[full]"

DOCX/PPTX conversion also needs weasyprint's native Pango/GObject/Cairo libraries, which pip cannot install for you:

# macOS
brew install pango

# Debian/Ubuntu (24.04 and older)
sudo apt-get install libpango-1.0-0 libpangocairo-1.0-0 libgdk-pixbuf2.0-0 libffi-dev shared-mime-info

# Debian trixie (13) and newer: libgdk-pixbuf2.0-0 was renamed
sudo apt-get install libpango-1.0-0 libpangocairo-1.0-0 libgdk-pixbuf-2.0-0 libffi-dev shared-mime-info

PDF conversion (the base install) does not need this.

OCR and equations need one native binary

Found by testing docmd against a real scanned document, not documented anywhere upstream at the time of writing: Marker's current OCR and equation-recognition model is a vision-language model served through either vllm (GPU/Linux-oriented) or llama.cpp's llama-server binary - there is no plain-CPU/transformers fallback. pip install docmd-cli cannot provide either one, since neither ships as a normal Python wheel.

This only matters for scanned PDFs (no embedded text layer) and PDFs with equations - a normal text-layer PDF never touches this code path, and everything else in this README works with just pip install.

# macOS / Linux with Homebrew
brew install llama.cpp

# No Homebrew: download a prebuilt binary directly, no package manager needed
# (pick the archive matching your OS/arch from the releases page)
curl -LO https://github.com/ggml-org/llama.cpp/releases/latest/download/llama-<version>-bin-macos-arm64.tar.gz
tar xzf llama-<version>-bin-macos-arm64.tar.gz
export LLAMA_CPP_BINARY=$PWD/llama-<version>/llama-server

Without it, converting a scanned PDF or one with equations raises a MissingSystemDependencyError with these same instructions - not a raw stack trace. First real OCR run also downloads the model's GGUF weights from Hugging Face (a few GB), separate from the PyTorch weights Marker already downloaded.

OCR output is not guaranteed reproducible run to run - confirmed directly: the same scanned file through the same code path twice produced different text. This is architectural (concurrent batched inference under the hood), not a docmd bug and not fixable by a config flag alone - see CONTRACT.md for the full investigation. Doesn't affect plain text-layer PDFs, only pages that actually go through OCR or equation recognition.

Office file limits

Tested against 14 real DOCX/PPTX files (Apache POI's public test set): 95-100% of the source text ends up in the output. Three things to know:

• Speaker notes are extracted separately from Marker, since its PowerPoint provider ignores them entirely - read directly from the source file with python-pptx and appended under their slide's heading as "Speaker notes:" (Markdown) or their own chunk with section="Slide N" (--format rag).
• page in chunks is a rendered page, not a Word page or a slide. DOCX and PPTX are rendered to PDF first, so page breaks are the renderer's. For a deck, use the Slide N section in a chunk's section, not page.
• WMF images couldn't be decoded in testing on macOS (cannot find loader for this WMF file). Logged as an error, non-fatal, the image is just absent.

Image handling

Images default to a text placeholder (*[... omitted]*) - no binary data, nothing to resolve, safe for RAG chunking:

markdown = convert("report.pdf")  # image_mode="placeholder" by default

To keep real image links instead, use image_mode="alt-text" and pass output_dir so the image files actually get saved somewhere the links can resolve to:

from docmd import convert_document
from docmd.config import ConvertConfig

result = convert_document(
    "report.pdf",
    config=ConvertConfig(image_mode="alt-text"),
    output_dir="output/",
)

From the CLI, --image-dir defaults to the output file's directory when -o is given:

docmd convert report.pdf -o output/report.md --image-mode alt-text

image_mode="skip" drops images entirely - no placeholder, no files.

Provenance

Every conversion carries tracking info in result.provenance - useful for "this converted differently yesterday" debugging:

from docmd import convert_document

result = convert_document("report.pdf")
print(result.provenance)
# {'docmd_version': '0.1.3', 'backend': 'marker', 'backend_version': '2.0.0',
#  'ocr_used': False, 'conversion_duration_ms': 489}

Same shape regardless of which backend actually ran - convert() (the plain string-returning function) doesn't expose this; use convert_document() for it.

RAG-ready chunks

For search/RAG use cases that need more than one Markdown string, opt into per-block chunks with page, section, content type, and bounding-box metadata:

from docmd import convert_document
from docmd.config import ConvertConfig

result = convert_document("report.pdf", config=ConvertConfig(include_chunks=True))
for chunk in result.chunks:
    print(chunk.page, chunk.content_type, chunk.section, chunk.text[:50])

Or from the CLI: docmd convert report.pdf --format rag -o chunks.json.

One chunk per structural block the backend identified (a paragraph, a table, a heading, an image, ...) by default. content_type is one of a small, stable set (text, heading, table, image, list), independent of Marker's own internal block-type names. section is a breadcrumb of the nearest heading at each level above the chunk (e.g. "Chapter 3 > 3.1 Introduction"). Chunk text is the block's own raw text, extracted before docmd's Markdown post-processing runs - a table chunk's text is the backend's flattened cell text, not a cleaned Markdown table. chunks is None unless include_chunks=True - the default convert_document() call is unaffected.

Raw blocks are often too small (a bare heading, a one-sentence paragraph) or too large (a big table) for good embeddings. Set chunk_max_tokens to merge adjacent text/heading chunks - same page, same section - up to roughly that many tokens (a cheap ~4-chars/token estimate, not a real tokenizer):

config = ConvertConfig(include_chunks=True, chunk_max_tokens=400)

Tables, images, and lists are never merged into surrounding text, and merging never crosses a page or section boundary. From the CLI: docmd convert report.pdf --format rag --chunk-max-tokens 400 -o chunks.json.

Extracting tables as CSV

Independent of the Markdown output, pull every table out of a document as its own CSV file:

docmd extract report.pdf --tables csv -o tables/
# wrote 3 table(s) to tables/  (table_1.csv, table_2.csv, table_3.csv)

Markdown's backslash-escaping (e.g. \$ 5,428, so a dollar figure isn't read as a math delimiter) is undone in the CSV output - that escaping is correct in Markdown but meaningless once re-purposed as CSV.

Validating the output

docmd validate report.pdf
✓ 12 heading(s) detected, no level skips
✓ 3 table(s), all consistent column counts

Reports concrete, verifiable findings about the converted document - never an invented quality percentage, since docmd has no ground truth to back one up with. Exits non-zero if anything is flagged (⚠), so it's usable as a CI gate on your own document pipeline. The checks are the same ones that guard docmd's own contract - see CONTRACT.md and docmd/validate.py.

Converting a whole directory

docmd batch ./documents -o ./converted
[1/247] converting: reports/q3.pdf
[2/247] skip (already converted): reports/q2.pdf
...
241 succeeded, 2 failed, 4 skipped (of 247 total)

Mirrors INPUT_DIR's structure into the output directory. A file whose output already exists is skipped, not redone - so an interrupted batch resumes cleanly by re-running the exact same command (--overwrite forces reconversion). One bad file doesn't stop the rest: failures are collected and reported at the end, with the command exiting non-zero if any occurred. Takes the same --force-ocr, --image-mode, --format, and --chunk-max-tokens options as docmd convert.

Runs sequentially, not in parallel - Marker's models are loaded once and reused across files in-process; concurrent conversions against that shared state haven't been verified safe, so this doesn't guess past that.

Searching across documents

docmd batch ./documents -o ./chunks --format rag
docmd search ./chunks "revenue in 2024"

annual-report.json
  Page 87
  Section: Financial Results
  "...revenue increased by..."

Searches the RAG chunk JSON docmd batch --format rag already produced - not raw documents - so repeated searches don't reconvert anything. Keyword matching ranked with BM25 (rare words weigh more than common ones, repeats saturate, long chunks are length-normalized) plus an exact-phrase bonus, not semantic search: a query for "revenue" won't find a chunk that only says "income". Semantic search is real future scope once there's a provider key to build and test an embedding-based mode against, the same way use_llm is gated on Marker's own LLM provider setup.

How it works

docmd wraps Marker with sane defaults and a clean output format, then runs its own post-processing pass (docmd/postprocess/) to fix table structure and normalize heading levels — see ARCHITECTURE.md for why this is the actual differentiation, not just a thin wrapper, and CONTRACT.md for exactly what's guaranteed about the output (and what isn't, yet) — independent of Marker or any future backend.

License

The docmd wrapper code is MIT — see LICENSE.

docmd depends on Marker, whose code and model weights carry separate licenses — the weights license has a revenue/funding threshold that matters if you deploy commercially at scale. See docs/licensing.md for the precise terms; don't rely on this sentence alone.

Roadmap

• PDF, DOCX, PPTX → Markdown
• Table cleanup / heading normalization post-processing
• Hosted API
• OCR quality tuning for scanned documents
• Batch conversion endpoint
• HTML output option

Contributing

Issues and PRs welcome. If you're hitting a conversion quality issue, please include a sample file (or a minimal reproduction) — it makes fixes much faster.

If docmd saves you the trouble of setting up your own PDF-parsing pipeline, consider starring the repo — it's how other people find it.