pdflite

pdflite turns a PDF into markdown that a person or a language model can read. The conversion itself is done by pdf-oxide, which reads the text stored in the PDF. That text comes out wrong in a few common ways, and pdflite fixes them:

• A word split across two lines by a hyphen, such as “docu-” and “ment”, is joined back into one word.
• A table laid out in two columns, such as a bank statement, can come out with its amounts on lines of their own, far from the labels they belong to. pdflite then tries the PDF’s plain-text layer for that page and keeps it if it leaves fewer amounts on their own.
• A scanned page has no text stored in it, only a picture of the page. pdflite sends such pages to OCR, optical character recognition, which reads the text from the picture.
• A web page printed to PDF from Medium carries “Press enter or click to view image” lines from the site’s reader, which are removed.

pdf_md does all of this in one call:

from pdflite import pdf_md

md = pdf_md('pdfs/attention_is_all_you_need.pdf', out_path='out'); md[:100] # out_path stores images
# pages = pdf_md('statement.pdf', out_path='out', pages=True)  # one entry per page

Dictionary used where Stream expected, treating as empty stream
Dictionary used where Stream expected, treating as empty stream
Dictionary used where Stream expected, treating as empty stream
Dictionary used where Stream expected, treating as empty stream

'Provided proper attribution is provided, Google hereby grants permission to reproduce the tables and'

out_path is the folder where images found in the PDF are saved, in an images folder inside it; the markdown links to them by relative paths. With pages=True, pdf_md returns a list with one string per page instead of one string, which is what you want when citing page numbers.

Each repair is also exported on its own, so you can use it on markdown from elsewhere: clean_md joins hyphenated words and tidies whitespace, orphan_vals and scrambled_layout detect a reflowed table, fix_layout repairs one, and needs_ocr, thin_text and ocr_parse handle scanned pages. oxide_parse runs the conversion without OCR, and pdf_pages reports how each page was read.

clean_md('a docu-\nment')      # a hyphen split across lines

'a document'

orphan_vals('Opening balance\nFees\n1,240.00\n18.50')   # values cut loose from their labels

2

Install

pip install pdflite

OCR needs liteparse, which is not installed with pdflite: run pip install liteparse if your PDFs include scanned pages. pdflite imports it only when a page needs OCR, and raises an ImportError naming the package if it is missing.