Skip to main content

Sparrow Parse is a Python package (part of Sparrow) for parsing and extracting information from documents.

Project description

Sparrow Parse

Description

This module implements Sparrow Parse library library with helpful methods for data pre-processing, parsing and extracting information. This library relies on Visual LLM functionality, Table Transformers and is part of Sparrow. Check main README

Install

pip install sparrow-parse

Parsing and extraction

Sparrow Parse VL (vision-language model) extractor with Hugging Face GPU infra

from sparrow_parse.vllm.inference_factory import InferenceFactory
from sparrow_parse.extractors.vllm_extractor import VLLMExtractor

extractor = VLLMExtractor()

# export HF_TOKEN="hf_"
config = {
    "method": "huggingface",  # Could be 'huggingface' or 'local_gpu'
    "hf_space": "katanaml/sparrow-qwen2-vl-7b",
    "hf_token": os.getenv('HF_TOKEN'),
    # Additional fields for local GPU inference
    # "device": "cuda", "model_path": "model.pth"
}

# Use the factory to get the correct instance
factory = InferenceFactory(config)
model_inference_instance = factory.get_inference_instance()

input_data = [
    {
        "image": "/data/bonds_table.png",
        "text_input": "retrieve all data. return response in JSON format"
    }
]

# Now you can run inference without knowing which implementation is used
result = extractor.run_inference(model_inference_instance, input_data, generic_query=False, debug=True)
print("Inference Result:", result)

PDF pre-processing

from sparrow_parse.extractor.pdf_optimizer import PDFOptimizer

pdf_optimizer = PDFOptimizer()

num_pages, output_files, temp_dir = pdf_optimizer.split_pdf_to_pages(file_path,
                                                                     output_directory,
                                                                     convert_to_images)

Example:

file_path - /data/invoice_1.pdf

output_directory - set to not None, for debug purposes only

convert_to_images - default False, to split into PDF files

Library build

Create Python virtual environment

python -m venv .env_sparrow_parse

Install Python libraries

pip install -r requirements.txt

Build package

pip install setuptools wheel
python setup.py sdist bdist_wheel

Upload to PyPI

pip install twine
twine upload dist/*

Commercial usage

Sparrow is available under the GPL 3.0 license, promoting freedom to use, modify, and distribute the software while ensuring any modifications remain open source under the same license. This aligns with our commitment to supporting the open-source community and fostering collaboration.

Additionally, we recognize the diverse needs of organizations, including small to medium-sized enterprises (SMEs). Therefore, Sparrow is also offered for free commercial use to organizations with gross revenue below $5 million USD in the past 12 months, enabling them to leverage Sparrow without the financial burden often associated with high-quality software solutions.

For businesses that exceed this revenue threshold or require usage terms not accommodated by the GPL 3.0 license—such as integrating Sparrow into proprietary software without the obligation to disclose source code modifications—we offer dual licensing options. Dual licensing allows Sparrow to be used under a separate proprietary license, offering greater flexibility for commercial applications and proprietary integrations. This model supports both the project's sustainability and the business's needs for confidentiality and customization.

If your organization is seeking to utilize Sparrow under a proprietary license, or if you are interested in custom workflows, consulting services, or dedicated support and maintenance options, please contact us at abaranovskis@redsamuraiconsulting.com. We're here to provide tailored solutions that meet your unique requirements, ensuring you can maximize the benefits of Sparrow for your projects and workflows.

Author

Katana ML, Andrej Baranovskij

License

Licensed under the GPL 3.0. Copyright 2020-2024 Katana ML, Andrej Baranovskij. Copy of the license.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sparrow-parse-0.3.5.tar.gz (14.0 kB view details)

Uploaded Source

Built Distribution

sparrow_parse-0.3.5-py3-none-any.whl (14.9 kB view details)

Uploaded Python 3

File details

Details for the file sparrow-parse-0.3.5.tar.gz.

File metadata

  • Download URL: sparrow-parse-0.3.5.tar.gz
  • Upload date:
  • Size: 14.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.4

File hashes

Hashes for sparrow-parse-0.3.5.tar.gz
Algorithm Hash digest
SHA256 a5d40011c035059b2cf344583d0d56d2c9807aac2fdcb5d58c65ad24954820da
MD5 364cd9a87b5905fab322f1393fa493ba
BLAKE2b-256 8fee7de78010a2a5271c95f99a12617277805932307f434c8b514d6f5e377d92

See more details on using hashes here.

File details

Details for the file sparrow_parse-0.3.5-py3-none-any.whl.

File metadata

File hashes

Hashes for sparrow_parse-0.3.5-py3-none-any.whl
Algorithm Hash digest
SHA256 3fc49368a0b2be40ab6121fd9e251af88d07c9b4b5100f0ac6fdf697d5f9d5b3
MD5 383d1d37cf2cc9aa5066a139bea86784
BLAKE2b-256 f7ebe672b1fc44b4c36d1d39b7702ffb3df3852f6b4deb1a8688258317bf5841

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page