Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.8.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | cb2d7cdff3175abb7a48a05a977fd3061cde9e6c8e1ac49c0d371dc6df159f4a |
|
MD5 | bc5bfff0fd45a7b376282ea38d56fe0f |
|
BLAKE2b-256 | b40cb1808b5def5a618da6e531d085efe0773d70bdd847aa850bbd1928eedc73 |
Close
Hashes for scrapysplashwrapper-1.8.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 396871639fe5c2d910dfe7e0f3fe425135143f9c8e1f3ab9d95d50375c4ea264 |
|
MD5 | 1f54353d933afc3c4a23209f565f42c7 |
|
BLAKE2b-256 | e68db430906fe68839d4ae24fc5febdd94ca3d9cd1ebf01685f48ef3988cc3b1 |