Spark Pdf

Spark-Pdf is a library for processing documents using Apache Spark.

It includes the following features:

• Load PDF documents/Images
• Extract text from PDF documents/Images
• Extract images from PDF documents
• OCR Images/PDF documents
• Run NER on text extracted from PDF documents/Images
• Visualize NER results

Installation

Requirements

• Python 3.10
• Apache Spark 3.5 or higher
• Java 8
• Tesseract 4.0 or higher

  pip install pyspark-pdf

Development

Setup

  git clone
  cd spark-pdf

Install dependencies

  poetry install

Run tests

  poetry run pytest --cov=sparkpdf --cov-report=html:coverage_report tests/ 

Build package

  poetry build

Build documentation

  poetry run sphinx-build -M html source build

Docker

Build image:

  docker build -t spark-pdf .

Run container:

  docker run --rm -it --entrypoint bash spark-pdf:latest

Release

  poetry version patch
  poetry publish --build