Skip to main content

PyMuPDF Utilities for LLM/RAG

Project description

Using PyMuPDF as Data Feeder in LLM / RAG Applications

This package converts the pages of a PDF to text in Markdown format using PyMuPDF.

Standard text and tables are detected, brought in the right reading sequence and then together converted to GitHub-compatible Markdown text.

Header lines are identified via the font size and appropriately prefixed with one or more '#' tags.

Bold, italic, mono-spaced text and code blocks are detected and formatted accordingly. Similar applies to ordered and unordered lists.

By default, all document pages are processed. If desired, a subset of pages can be specified by providing a list of 0-based page numbers.

Installation

$ pip install -U pymupdf4llm

This command will automatically install PyMuPDF if required.

Then in your script do:

import pymupdf4llm

md_text = pymupdf4llm.to_markdown("input.pdf")

# now work with the markdown text, e.g. store as a UTF8-encoded file
import pathlib
pathlib.Path("output.md").write_bytes(md_text.encode())

Instead of the filename string as above, one can also provide a PyMuPDF Document. By default, all pages in the PDF will be processed. If desired, the parameter pages=[...] can be used to provide a list of zero-based page numbers to consider.

Feature Overview:

  • Support for pages with multiple text columns.

  • Support for image and vector graphics extraction:

    1. Specify pymupdf4llm.to_markdown("input.pdf", write_images=True). Default is False.
    2. Each image or vector graphic on the page will be extracted and stored as an image named "input.pdf-pno-index.extension" in a folder of your choice. The image extension can be chosen to represent a PyMuPDF-supported image format (for instance "png" or "jpg"), pno is the 0-based page number and index is some sequence number.
    3. The image files will have width and height equal to the values on the page. The desired resolution can be chosen via parameter dpi (default: dpi=150).
    4. Any text contained in the images or graphics will be extracted and also become visible as part of the generated image. This behavior can be changed via force_text=False (text only apears as part of the image).
  • Support for page chunks: Instead of returning one large string for the whole document, a list of dictionaries can be generated: one for each page. Specify data = pymupdf4llm.to_markdown("input.pdf", page_chunks=True). Then, for instance the first item, data[0] will contain a dictionary for the first page with the text and some metadata.

  • As a first example for directly supporting LLM / RAG consumers, this version can output LlamaIndex documents:

    import pymupdf4llm
    
    md_read = pymupdf4llm.LlamaMarkdownReader()
    data = md_read.load_data("input.pdf")
    
    # The result 'data' is of type List[LlamaIndexDocument]
    # Every list item contains metadata and the markdown text of 1 page.
    
    • A LlamaIndex document essentially corresponds to Python dictionary, where the markdown text of the page is one of the dictionary values. For instance the text of the first page is the the value of data[0].to_dict().["text"].
    • For details, please consult LlamaIndex documentation.
    • Upon creation of the LlamaMarkdownReader all necessary LlamaIndex-related imports are executed. Required related package installations must have been done independently and will not be checked during pymupdf4llm installation.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pymupdf4llm-0.0.22.tar.gz (28.1 kB view details)

Uploaded Source

Built Distribution

pymupdf4llm-0.0.22-py3-none-any.whl (28.7 kB view details)

Uploaded Python 3

File details

Details for the file pymupdf4llm-0.0.22.tar.gz.

File metadata

  • Download URL: pymupdf4llm-0.0.22.tar.gz
  • Upload date:
  • Size: 28.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.3

File hashes

Hashes for pymupdf4llm-0.0.22.tar.gz
Algorithm Hash digest
SHA256 b63a0fbbf3b1b1b42f239e2e23797f3b25d2163470dc7f4f84841b7bf3618c2c
MD5 3bb688982100249e675ccb1d4e4fd75b
BLAKE2b-256 e6f439f041220a7e328f4c71cc72953e3ff29ed188396b2ba1b45daca235f50f

See more details on using hashes here.

File details

Details for the file pymupdf4llm-0.0.22-py3-none-any.whl.

File metadata

  • Download URL: pymupdf4llm-0.0.22-py3-none-any.whl
  • Upload date:
  • Size: 28.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.3

File hashes

Hashes for pymupdf4llm-0.0.22-py3-none-any.whl
Algorithm Hash digest
SHA256 80716a74e0895d8a56a2edaad02484308ea93bd8aa80b8ef3d3d90e4e49ce8b6
MD5 b44820e2dfbbe04298f390eeae13c105
BLAKE2b-256 d6ab0f3f2f6a6d8521cfdb8226f63a50d22e7c6ca5e2fb72f1fbf56efda04218

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page