Using PyMuPDF as Data Feeder in LLM / RAG Applications
This package converts the pages of a PDF to text in Markdown format using PyMuPDF.
Standard text and tables are detected, brought in the right reading sequence and then together converted to GitHub-compatible Markdown text.
Header lines are identified via the font size and appropriately prefixed with one or more '#' tags.
Bold, italic, mono-spaced text and code blocks are detected and formatted accordingly. Similar applies to ordered and unordered lists.
By default, all document pages are processed. If desired, a subset of pages can be specified by providing a list of 0-based page numbers.
Installation
$ pip install -U pdf4llm
This command will automatically install PyMuPDF if required.
Then in your script do:
import pdf4llm
md_text = pdf4llm.to_markdown("input.pdf")
# now work with the markdown text, e.g. store as a UTF8-encoded file
import pathlib
pathlib.Path("output.md").write_bytes(md_text.encode())
Instead of the filename string as above, one can also provide a PyMuPDF Document. By default, all pages in the PDF will be processed. If desired, the parameter pages=[...] can be used to provide a list of zero-based page numbers to consider.
New features as of v0.0.8:
-
Support for pages with multiple text columns.
-
Support for image and vector graphics extraction:
- Specify
pdf4llm.to_markdown("input.pdf", write_images=True). Default isFalse. - Each image or vector graphic on the page will be extracted and stored as a PNG image named
"input.pdf-pno-index.png"in the folder of"input.pdf". Wherepnois the 0-based page number andindexis some sequence number. - The image files will have width and height equal to the values on the page.
- Any text contained in the images or graphics will not be extracted, but become visible as image parts.
- Specify
-
Support for page chunks: Instead of returning one large string for the whole document, a list of dictionaries can be generated: one for each page. Specify
data = pdf4llm.to_markdown("input.pdf", page_chunks=True). Then, for instance the first item,data[0]will contain a dictionary for the first page with the text and some metadata. -
As a first example for directly supporting LLM / RAG consumers, this version can output LlamaIndex documents:
import pdf4llm md_read = pdf4llm.LlamaMarkdownReader() data = md_read.load_data("input.pdf") # The result 'data' is of type List[LlamaIndexDocument] # Every list item contains metadata and the markdown text of 1 page.
- A LlamaIndex document essentially corresponds to Python dictionary, where the markdown text of the page is one of the dictionary values. For instance the text of the first page is the the value of
data[0].to_dict().["text"]. - For details, please consult LlamaIndex documentation.
- Upon creation of the
LlamaMarkdownReaderall necessary LlamaIndex-related imports are executed. Required related package installations must have been done independently and will not be checked during pdf4llm installation.
- A LlamaIndex document essentially corresponds to Python dictionary, where the markdown text of the page is one of the dictionary values. For instance the text of the first page is the the value of
4 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pdf4llm-1.28.2.tar.gz.
File metadata
- Download URL: pdf4llm-1.28.2.tar.gz
- Upload date:
- Size: 15.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
fc81f484effdfd4f55ad57146a8ef8332fb6f0c03e3209095427fb28063d11ca
|
|
| MD5 |
1dc6b02d43a4ee5301bb491919bc4cd7
|
|
| BLAKE2b-256 |
1bb132107ce8d116711b8ba513670265e8c9516aa01be81a64ccf8f271e20941
|
File details
Details for the file pdf4llm-1.28.2-py3-none-any.whl.
File metadata
- Download URL: pdf4llm-1.28.2-py3-none-any.whl
- Upload date:
- Size: 15.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f7b6b6bd716e2aaa2b1f604f838810848ca7eed444496ce141b3c11b3c9e0f4f
|
|
| MD5 |
4601deb17aa4bc0576728916e03f48ae
|
|
| BLAKE2b-256 |
122533a8aed65aa1b8fbd819eaf1d6b206bd53e5bdfcc10957ad888d13fb2022
|
