A web scraper that combines both Beautiful Soup (bs4) and Selenium.
Want to support the development and stay updated?
Installation
pip install PyWebScraper
Usage
from PyWebScraper import Scraper
Scraper() will load bs4 or selenium (defined with the input 'scraper_type') to then load a website (defined by the input 'url') and save it under Scraper().page.
Optional inputs for Scraper():
url = str (will be opened in scraper and page saved in Scraper().page)
scraper_type = 'bs4' or 'selenium'
scroll_down = boolean (scrolls down in selenium first before saving page)
user_agent = 'desktop' or 'mobile'
auto_close_selenium = boolean (if False, you can further interact with the selenium browser via Scraper().selenium)
selenium_remote_webdriver = str (IP for a remote webdriver for selenium, see https://www.selenium.dev/docs/site/en/remote_webdriver/remote_webdriver_client/)
1 maintainer
Metadata
1 maintainer
Release files for PyWebScraper 0.1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| PyWebScraper-0.1.5.tar.gz | 4.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| PyWebScraper-0.1.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 11.2 kB
Release files / PyWebScraper-0.1.5.tar.gz
| Download URL | PyWebScraper-0.1.5.tar.gz |
|---|---|
| Size | 4.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f1b829b4aa796ebe3b4da753eb43286ff2d15ab928b42539cc6258f58ad1012c
|
|
BLAKE2b-256 checksum How to use checksums |
5983e2abdc5ea9f56d77b37dc8473b130799ddf117e6b63dc0360d4d3dff8592
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.8.2
|
Release files / PyWebScraper-0.1.5-py3-none-any.whl
| Download URL | PyWebScraper-0.1.5-py3-none-any.whl |
|---|---|
| Size | 6.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
41a147d365d9803a27d7c1e37eea3c713ab2d1a6f5a4e557054ab69f7b68a2fe
|
|
BLAKE2b-256 checksum How to use checksums |
2849d2b68f4d8047ab83b8afe067a4d1d89ad6d327f07beff797c326d45ba87a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.8.2
|
