llama-index-readers-pdfluent

A LlamaIndex reader for PDFluent. It reads a PDF with PDFluent's structured extraction and hands LlamaIndex Documents in reading order, with the page and box each came from in its metadata so an answer can cite its source. Everything runs locally. Your documents stay on your machine.

Install

pip install llama-index-readers-pdfluent

It needs Python 3.9 or later and pdfluent 1.2.0 or later, which pip pulls in automatically. It also depends on llama-index-core (MIT).

Quick start

from llama_index.readers.pdfluent import PDFluentReader

docs = PDFluentReader().load_data("report.pdf")
print(docs[0].text, docs[0].metadata["page"])

A runnable notebook, PDF to VectorStoreIndex in about ten lines with a local embedding and no API key, is in examples/pdf_to_index.ipynb.

Modes

mode decides what one Document is.

Blocks mode

One document per paragraph, heading, list item or table (a table is one document), in reading order. This is the default.

from llama_index.readers.pdfluent import PDFluentReader

reader = PDFluentReader()
for doc in reader.lazy_load_data("report.pdf"):
    print(doc.metadata["kind"], doc.metadata["page"], doc.metadata["bbox"])

Markdown mode

One document per page, as markdown. Pass quality="balanced" for the layout-model tier (see below).

from llama_index.readers.pdfluent import PDFluentReader

pages = PDFluentReader(mode="markdown").load_data("report.pdf")
print(len(pages), "pages")
print(pages[0].text[:200])

Document mode

One document for the whole selection: the markdown of every selected page, joined by a blank line. metadata["pages"] lists the pages it covers.

from llama_index.readers.pdfluent import PDFluentReader

whole = PDFluentReader(mode="document", pages=[0, 2]).load_data("report.pdf")
print(whole[0].metadata["pages"])

Quality and pages

quality is "fast" (default) or "balanced", the layout-model tier of pdfluent 1.2. The balanced tier assembles markdown, so it goes with mode="markdown" or mode="document"; with mode="blocks" the engine refuses it and says so. It needs the model weights, installed once with the pdfluent command-line tool:

pdfluent models install layout tables

from llama_index.readers.pdfluent import PDFluentReader

PDFluentReader(mode="markdown", quality="balanced").load_data("report.pdf")

pages=[0, 3] loads exactly those pages (0-based); first_page and page_count load a window.

Every PDF in a directory

from llama_index.core import SimpleDirectoryReader
from llama_index.readers.pdfluent import PDFluentReader

documents = SimpleDirectoryReader(
    "pdfs/", file_extractor={".pdf": PDFluentReader()}
).load_data()

Options

Metadata

bbox, toc_items and tables are lists. The geometry and the confidence are for citing and highlighting, so they are excluded from what the embedding model and the LLM see; source, page, kind and section stay in.

Benchmarks

Extraction quality is measured in the public pdfluent-bench repository (link follows when it is public).

Licence

AGPL-3.0 or a commercial licence, at your option; see LICENSE and LICENSE-COMMERCIAL. The commercial licence is sold yearly and self-service at https://pdfluent.com/sdk/pricing.