An open-source Python library for web scraping tasks. Includes support for both image scraping and text scraping.
Project description
PyWebScrapr
An open-source Python library for web scraping tasks. Includes support for both text and image scraping.
Changes in 0.1.2
Changes in version 0.1.2:
min
andmax
width and height parameters can now be specified when working with image scraping, allowing you to quickly exclude smaller resolution images, or images that are extremely large and take up too much space.- PyWebScrapr now uses BeautifulSoup4's
SoupStrainer
, making extracting content from webpages much faster.
Installation
You can install PyWebScrapr using pip:
pip install pywebscrapr
Supported Python Versions
PyWebScrapr supports the following Python versions:
- Python 3.6
- Python 3.7
- Python 3.8
- Python 3.9
- Python 3.10
- Python 3.11
- Python 3.12/Later (Preferred)
Please ensure that you have one of these Python versions installed before using PyWebScrapr. PyWebScrapr may not work as expected on lower versions of Python than the supported.
Features
- Text Scraping: Extract textual content from specified URLs.
- Image Scraping: Download images from specified URLs.
*for a full list check out the PyWebScrapr official documentation.
Usage
Text Scraping
from pywebscrapr import scrape_text
# Specify links in a file or list
links_file = 'links.txt'
links_array = ['https://example.com/page1', 'https://example.com/page2']
# Scrape text and save to the 'output.txt' file
scrape_text(links_file=links_file, links_array=links_array, output_file='output.txt')
Image Scraping
from pywebscrapr import scrape_images
# Specify links in a file or list
links_file = 'image_links.txt'
links_array = ['https://example.com/image1.jpg', 'https://example.com/image2.png']
# Scrape images and save to the 'images' folder
scrape_images(links_file=links_file, links_array=links_array, save_folder='images')
Contributing
Contributions are welcome! If you encounter any issues, have suggestions, or want to contribute to PyWebScrapr, please open an issue or submit a pull request on GitHub.
License
PyWebScrapr is released under the terms of the MIT License (Modified). Please see the LICENSE file for the full text.
Modified License Clause
The modified license clause grants users the permission to make derivative works based on the PyWebScrapr software. However, it requires any substantial changes to the software to be clearly distinguished from the original work and distributed under a different name.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pywebscrapr-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f50fa43875443177be524f0f538549e1824c607855487dfc414847cceea80ff |
|
MD5 | bfc10a2a793178ad7b2831bfe11687c8 |
|
BLAKE2b-256 | 281d95c45090f591395ef539d3aa4760b1118b2e16c4152282e1693dea3f8f9a |