llama-index-readers-pdfluent

A LlamaIndex reader for PDFluent. It reads a PDF with PDFluent's structured extraction and hands LlamaIndex one Document per paragraph, heading, list item or table, in reading order, with the page and box each came from in its metadata so an answer can cite its source.

Install

pip install llama-index-readers-pdfluent

It needs Python 3.9 or later and pdfluent 1.1.0 or later, which pip pulls in automatically. It also depends on llama-index-core (MIT).

Use

Blocks mode

One document per block, a table as one document, in reading order:

from llama_index.core import VectorStoreIndex
from llama_index.readers.pdfluent import PDFluentReader

documents = PDFluentReader().load_data("report.pdf")
index = VectorStoreIndex.from_documents(documents)

print(documents[0].text)
print(documents[0].metadata["page"])
print(documents[0].metadata["bbox"])
print(documents[0].metadata["kind"])

Markdown mode

One document per page, as markdown:

from llama_index.core import VectorStoreIndex
from llama_index.readers.pdfluent import PDFluentReader

documents = PDFluentReader(mode="markdown").load_data("report.pdf")
index = VectorStoreIndex.from_documents(documents)

print(documents[0].text[:200])
print(documents[0].metadata["page"])
print(documents[1].metadata["page"])
print(documents[1].text[:200])

Every PDF in a directory

from llama_index.core import SimpleDirectoryReader
from llama_index.readers.pdfluent import PDFluentReader

documents = SimpleDirectoryReader(
    "./reports", file_extractor={".pdf": PDFluentReader()}
).load_data()

Options:

Metadata

bbox, confidence, level and continues_previous are excluded from the text the embedding model and the LLM see; they are for citing and highlighting. extra_info (and the file metadata SimpleDirectoryReader adds) is merged into every document.

Licence

AGPL-3.0 or a commercial licence, at your option; see LICENSE and LICENSE-COMMERCIAL. The commercial licence is sold yearly and self-service at https://pdfluent.com/sdk/pricing.