Skip to main content

Python Version Contributions welcome License Build Status

Documentation Status PyPI - Downloads Stars

A python library which can be used to extraxct data from files, pdfs, doc(x) files, as well as save data into these files. This library can be used to scrape and extract webpage data from websites as well.

Installation Requirements and Instructions

Python versions 3.8 or above should be installed. After that open your terminal: For Windows users:

pip install dputils

For Mac/Linux users:

pip3 install dputils

Files Module

Functions from dputils.files: for now, the files module has two functions:

  1. get_data:

    • To import, use statement:

      from dputils.files import get_data
      
    • Obtains data from files of any extension given as args(supports text files, binary files, pdf, doc for now, more coming!)

    • sample call:

      content = get_data(r"sample.docx")
      print(content)
      
    • Returns a string or binary data depending on the output arg

    • images will not be extracted

  2. save_data:

    • save_data can be used to write and save data into a file of valid extension.
    • sample call:
      from dputils.files import save_data
      
      pdfContent = save_data("sample.pdf", "Sample text to insert")
      print(pdfContent)
      
    • Returns True if file is successfully accessed and modified. Otherwise, False.

Scrape Module

Data extraction from a page

Here's a basic tutorial to help you get started with the scraper module.

  1. Import the required classes and functions:
from dputils.scrape import Scraper, Tag
  1. Initialize the Scraper class with the URL of the webpage you want to scrape:
url = "https://www.example.com"
scraper = Scraper(url)
  1. Define the tags you want to scrape using the Tag class:
title_tag = Tag(name='h1', cls='title', output='text')
price_tag = Tag(name='span', cls='price', output='text')
  1. Extract data from the page:
data = scraper.get_data_from_page(title=title_tag, price=price_tag)
print(data)

Extracting list of items from a page

For more advanced usage, such as extracting repeated data from lists of items on a page, you can use the following approach:

  1. Initialize the Scraper class:
url = "https://www.example.com/products"
scraper = Scraper(url)
  1. Define the tags for the target section and the items within that section: For repeated data extraction, you need to define Target and item and pass it to get_repeating_data_from_page() method.
    • target - defines the Tag() for area of the page containing the list of items.
    • items - defines the Tag() for repeated items within the target section. Like a product-card in product grid/list.
target_tag = Tag(name='div', cls='product-list')
item_tag = Tag(name='div', cls='product-item')
title_tag = Tag(name='h2', cls='product-title', output='text')
price_tag = Tag(name='span', cls='product-price', output='text')
link_tag = Tag(name='a', cls='product-link', output='href')
  1. Extract repeated data from the page:
products = scraper.get_repeating_data_from_page(
    target=target_tag,
    items=item_tag,
    title=title_tag,
    price=price_tag,
    link=link_tag
)
for product in products:
    print(product)

These functions can used on python versions 3.8 or greater.

References for more help: https://digipodium.github.io/dputils/

Contribution

if you want to contribute to this project and make it better, your help is very welcome.

  • Fork the project
  • Create your feature branch (git checkout -b feature/fooBar)
  • Commit your changes (git commit -am 'Add some fooBar')
  • Push to the branch (git push origin feature/fooBar)
  • Create a new Pull Request
  • Wait for your PR to be reviewed and merged
  • Star the project if you've found it useful
  • Share the project with your friends
  • Create an issue if you find a bug or want to request a new feature
  • Improve the project by refactoring the code
  • Review the PRs of other contributors
  • Suggest new features
  • Suggest new technologies to be used

Thank you for using dputils!

Metadata

Release files for dputils 1.0.6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dputils 1.0.6
File Size Uploaded
dputils-1.0.6.tar.gz 8.1 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for dputils 1.0.6
File Interpreter ABI Platform
dputils-1.0.6-py3-none-any.whl Python 3 none any Details

Total release size: 17.2 kB

Release files / dputils-1.0.6.tar.gz

Download URL dputils-1.0.6.tar.gz
Size 8.1 kB
Tags Source
SHA-256 checksum
How to use checksums
12f378ddbc03b953197b8efc0257c3b60c3b8ad56ba8233a69d19adf2a81eae1
BLAKE2b-256 checksum
How to use checksums
94e0be01313d437ea4d5a51b11a702d35d0d4f171d6832902c253a97426f38a3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.3.2 CPython/3.13.9 Windows/11

Release files / dputils-1.0.6-py3-none-any.whl

Download URL dputils-1.0.6-py3-none-any.whl
Size 9.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
f33c1f893081f39f4af3368bcd5972062df5579cf1331dc91200273f8785ecd9
BLAKE2b-256 checksum
How to use checksums
04a372f128bd66e2295529eb393e40557084bf21b7e898b2a068be7249098510
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.3.2 CPython/3.13.9 Windows/11

Release history Release notifications | RSS feed

This release

1.0.6 This release

2 release files

1.0.5

2 release files

1.0.4

2 release files

1.0.3

2 release files

1.0.2

2 release files

1.0.1

2 release files

1.0.0

2 release files

0.3.0

2 release files

0.2.9

2 release files

0.2.8

2 release files

0.2.7

2 release files

0.2.6

2 release files

0.2.5

2 release files

0.2.4

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.15

2 release files

0.1.14

2 release files

0.1.13

2 release files

0.1.12

2 release files

0.1.11

2 release files

0.1.10

2 release files

0.1.9

2 release files

0.1.8

2 release files

0.1.7

2 release files

0.1.6

2 release files

0.1.5

2 release files

0.1.3

2 release files

0.1.2

2 release files

0.1.1

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page