pdf-to-md

Turn PDFs, scans, and Office documents into clean Markdown with one command, even 1,000-page books.

pdf-to-md is a command-line tool that converts documents to Markdown, HTML, or JSON with the Datalab Convert API, the hosted version of the open-source Marker engine. It can also transcribe PDFs and images with OpenAI's GPT-Luna, called directly or through OpenRouter. It handles the tedious parts for you: splitting big PDFs, uploading in parallel, retrying, stitching the results back together, and fixing page numbers and image links.

$ pdf-to-md books/
✓ darwin.pdf → darwin.md (516 pages, $1.55)
✓ tolstoy.pdf → tolstoy.md (537 pages, $1.61)
✓ grimm.pdf → grimm.md (510 pages, $1.53)
Done in 4m37s: 3 converted · 1563 pages · $4.69

_{Output from a real run (file names shortened): three scanned 19th-century books in English, Russian, and German Fraktur.}

Why pdf-to-md

• Built for big documents. PDFs are split into page chunks that convert in parallel and are merged back into a single file. Page separators, anchors, and image links stay correct across chunks.
• Safe to re-run. Output names are deterministic (report.pdf → report.md) and finished files are skipped. Re-running on a folder converts only what's new or failed, so you never pay twice by accident.
• Honest about results. Every file reports its pages and cost. The exit code says whether anything failed. A bad API key stops the run immediately instead of failing file by file.
• Resilient. Rate limits, server errors, and dropped connections are retried with backoff. Ctrl-C stops cleanly without leaving half-written files.
• Made for scripting. Converted paths go to stdout, progress and errors to stderr, so it drops straight into pipelines and CI jobs.
• Any language, any era. Tested on modern PDFs, math, statistical tables from a faded microfiche, and scans of Cyrillic, blackletter German, vertical classical Chinese, Arabic, Hebrew, polytonic Greek, Devanagari, and 1687 Latin (samples).

Installation

Requires Python 3.11 or newer. pipx installs it in its own isolated environment:

pipx install pdf-to-markdown-cli

uv tool install pdf-to-markdown-cli and pip install pdf-to-markdown-cli work too.

Then get an API key from datalab.to/app/keys and export it:

export DATALAB_API_KEY="your_api_key"

To use GPT-Luna instead, export OPENAI_API_KEY or OPENROUTER_API_KEY and pass --backend openai or --backend openrouter.

Quick start

pdf-to-md report.pdf

This writes report.md next to the PDF, plus a report_images/ folder if the document contains images. That's it.

Recipes

# Mirror a folder tree elsewhere
pdf-to-md ~/papers -o ~/papers-md

# Preview files and pages, at no cost
pdf-to-md ~/papers --dry-run

# Best quality for hard scans and tables
pdf-to-md scan.pdf --mode accurate

# First ten pages plus page 42 (0-based)
pdf-to-md book.pdf --page-range 0-9,42

# HTML or JSON instead of Markdown
pdf-to-md slides.pptx --html
pdf-to-md form.pdf --json

# Mark page boundaries in the output
pdf-to-md contract.pdf --paginate

# Text only: no images or descriptions
pdf-to-md manual.pdf --no-images \
  --no-image-captions

# Transcribe with GPT-Luna
pdf-to-md scan.pdf --backend openai
pdf-to-md scan.pdf --backend openrouter

# Redo everything, replacing old outputs
pdf-to-md ~/papers --overwrite

# Feed converted files to other tools
pdf-to-md ~/papers -q | xargs wc -w

How it works

1. Plan. Inputs are discovered recursively. Hidden files and folders this tool generated earlier are ignored. Each input gets a deterministic output path, and inputs whose output already exists are skipped.
2. Split. PDFs are cut into chunks of --chunk-size pages (default 25 for Datalab, 5 for GPT-Luna). Page selection with --page-range or --max-pages happens here, so only those pages are uploaded and billed. Other formats are uploaded whole.
3. Convert. Up to --concurrency chunks (default 5) are processed at once. Each one is uploaded, then polled until it's done (Datalab) or answered in one request (GPT-Luna). Throttling and server errors are retried, honoring the API's Retry-After.
4. Merge. Chunk results are joined in page order. Page separators, HTML page ids, JSON block ids, and in-document anchors are renumbered to match the original document. Images are renamed so they never collide, and links to them are URL-encoded.
5. Write. The output is written to a temporary file and renamed into place, so an interrupted run never leaves a partial file behind.

Output

papers/
├── report.pdf
├── report.md       ← the document
└── report_images/  ← images, if any

• If two inputs would produce the same name (a.pdf and a.docx), both keep their source extension: a.pdf.md and a.docx.md. Names are compared case-insensitively, and an output never overwrites an input.
• Hard-wrapped paragraph lines are joined in Markdown output (disable with --keep-line-breaks). Image descriptions live in the image alt text.

Supported formats

Outputs are Markdown (.md, the default), HTML (.html), or JSON (.json, Marker's block tree with page and position data). The API accepts files of up to 200 MB. Non-PDF files are always sent as a single request.

Quality, speed, and cost

--mode picks a Datalab processing mode:

Every result line shows what Datalab billed, and the run summary adds it up. In our September 2026 test runs, conversions cost about 0.3¢ per page ($3 per 1,000 pages). See Datalab's pricing for current rates.

Throughput depends on your plan's limits. With the default of 5 concurrent requests (the free tier's limit), the 1,563-page run above took 4 minutes 37 seconds. Paid plans allow far more concurrency, so raise -j to match yours.

Transcribing with GPT-Luna

--backend openai sends each chunk to OpenAI's GPT-6 Luna, a low-cost vision model, through the Responses API. --backend openrouter sends the same request to the same model through OpenRouter. The model sees each page's image and its text layer and writes one transcription per page. The tool checks that it got exactly one page back for every page sent, then merges the pages the same way as Datalab results.

Datalab remains the default. GPT-Luna runs only when you choose it with --backend.

In September 2026 test runs, GPT-Luna transcribed all 25 sample pages for about 2¢ on either route. That's 0.08¢ per page on average and about 0.2¢ for the dense census tables, versus about 0.3¢ per page on Datalab. Both routes bill the same token rates, and results show the cost of each file. For half price in exchange for slower responses, add --api-option service_tier=flex (OpenAI only).

--mode maps to reasoning effort: fast is none, balanced (the default) is low, and accurate is medium. In balanced mode every sample passed its checks, including the 1880 microfiche table that needs Datalab's accurate mode. As with any language model, output can vary between runs: in repeated tests it occasionally dropped a heading label or a LaTeX backslash.

--model picks another model, such as gpt-5.6-luna. OpenRouter adds the openai/ prefix when the name has none, so the same flag works on both routes. --api-option KEY=VALUE adds any other Responses API field, with JSON values decoded, e.g. --api-option max_output_tokens=32000.

Keep chunks small. In testing, a 16-page request came back with pages split apart, while 5- and 8-page chunks were exact. If a chunk comes back with the wrong page count, it's retried once, then the file fails with a hint to lower --chunk-size.

Sample conversions

examples/ holds sample documents next to their actual output from this tool:

See examples/README.md for sources and how the samples are used in tests.

Command reference

pdf-to-md INPUT [INPUT ...] [options]

INPUT is any mix of files and directories. The tool also runs as python -m docs_to_md.

Output

Conversion

Pages and chunking

Run control

Scripting and automation

Only converted file paths are printed to stdout, one per line. Status lines, the progress bar, and errors go to stderr.

Because finished files are skipped, the simplest way to retry failures in an interrupted batch is to run the same command again.

Configuration

--api-key overrides whichever variable the chosen backend reads. The tool keeps no state between runs. Temporary chunk files live in the system temp directory and are deleted when the run ends.

Troubleshooting

Anything else: rerun with -v and open an issue with the output.

Upgrading from 0.x

Version 1.0 moved to Datalab's current /api/v1/convert endpoint and fixed --html and --json, which produced no output in 0.5.x.

• Outputs are now report.md + report_images/ instead of report_<random>.md + images_<random>/, and existing outputs are skipped unless you pass --overwrite.
• -o accepts relative paths and mirrors folder structure.
• --llm now means --mode balanced and --max means --mode accurate. Datalab removed the options behind --strip, --force, and --langs, so those flags are ignored with a warning.
• --pages, --noimg, -cs, and -mp still work as aliases of --paginate, --no-images, --chunk-size, and --max-pages.
• Failures now exit non-zero, and ~/.docs_to_md/ is no longer used and can be deleted.

Full details are in the changelog.

Contributing

Bug reports, ideas, and pull requests are welcome. CONTRIBUTING.md covers setup, the test suites (including the opt-in live API tests), and the release process.

Acknowledgements

Conversion quality comes from Marker and the Datalab API, or from OpenAI's GPT-Luna. This is an independent project and isn't affiliated with Datalab, OpenAI, or OpenRouter.

License

MIT © Nikita Sokolsky