Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.2.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 97452b1c4a0b819736cc62eb3d79387de8a3e40b877b6c3a483dde30939e4f66 |
|
MD5 | 6b3ee58d5bb1a3608c7b3c8e46a4cf0f |
|
BLAKE2b-256 | f3d057876b5ba22bf552ff1ea0f60c90887f1def4e9151c4767e3788030f8fb2 |
Close
Hashes for scrapysplashwrapper-1.2.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fa62793bf77e778aa24f753de5caa21d14cb3e66fed36493b9698a36b5f169bb |
|
MD5 | 17184c15c29830399ba04b4b6af91da1 |
|
BLAKE2b-256 | 3a381665b3365e8d938d4b9aeb1f816c4ad67597d9c30e19df9d9b6d89a0d496 |