Skip to main content

Efficient PDF analysis, text extraction, preprocessing, and pattern recognition with customizable configurations and utilities.

Project description

Extractlib

This is a Python package that provides a set of tools and utilities for processing and analyzing PDF documents. It includes functionality for extracting text and tables from PDFs, cleaning and preprocessing text data, and analyzing content for keywords and patterns. The package also provides a number of configuration options for customizing the behavior of the tools and utilities, making it flexible and easy to use in a variety of different contexts. Whether you need to extract data from PDF documents for data analysis, or analyze PDF content for specific keywords or patterns, this package provides the tools you need to get the job done quickly and efficiently.

Dependency Overview

Python Dependency Install

This project leverages packages

nltk == 3.8.1 
PyMuPDF == 1.21.1 
camelot-py == 0.11.0 
opencv-python == 4.7.0.72 
ghostscript == 0.7

manually install supporting binaries

camelot dependencies

Ubuntu

$ apt install ghostscript python3-tk

MacOS

$ brew install ghostscript tcl-tk

windows dependency installations

Configuration

  • configuration file should be placed in the root of the project and named 'extractlib.config.json'

Example 'extractlib.config.json' File

this file should be located in the root of your project

{
  "std_out_logging": true,
  "supported_file_types": [".pdf"],
  "invalid_content_regexs": ["X{2,}"],
  "stop_words": [
    "na",
    "dependent",
    "address",
    "plans",
    "network",
    "nonnetwork",
    "additional",
    "covered"
  ],
  "keywords": {
    "dental": 5,
    "vision": 5,    
    "life": 5,
    "disability": 5
  },
  "keyword_synonyms": {
    "dental": ["orthodontic", "Endo", "Perio", "Oral"],
    "vision": ["eye", "vision", "lens", "lenses", "contact", "contacts"],
    "life": [
      "accident",
      "critical",
      "illness",
      "accidental",
      "dismember",
      "AD&D"
    ],
    "disability": []
  },
  "word_min_length": 3
}

access config variables

from extractlib.settings import config

print(json.dump(config.config_raw, indent=4))

Example implementation

from extractlib.document.process import process_document
import json

def main(file: str):
    result = process_document(file,  exclude_pages=[2,3], use_multithreading=False, split_pages_output_dir='./output', delete_split_pages=False)
    # Save the HTML content to a temporary file
    with open('temp.json', 'w') as f:
        json.dump(result, f, indent=4)


if __name__ == '__main__':

    # get working directory
    import os
    target_dir = os.path.dirname(os.path.abspath(__file__))
    main(f'{target_dir}/_testdata/PDF.pdf')

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

insights-extractor-0.1.1.tar.gz (46.8 kB view hashes)

Uploaded Source

Built Distribution

insights_extractor-0.1.1-py3-none-any.whl (17.7 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page