Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.4.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0bf67cc23da6cfe1f7ad8a2c21480624b916834fb65db8324bf207ae22f5d671 |
|
MD5 | 599f873a003d96b44f458a3d86cd24f3 |
|
BLAKE2b-256 | 8b07226c283db63b347776d99491403039927a08810ba6ea68f6a8558c46ea17 |
Close
Hashes for scrapysplashwrapper-1.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8aa443a7bec5291983384213c7627c411c919bdd1570c0140aa219a2ab269b11 |
|
MD5 | 55929c0749b699760a410eb08074fa0e |
|
BLAKE2b-256 | e349ff389676bb2b8b588fac9d0331337584c5462e581f41995c22526a152a51 |