Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.6.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 46bfd2c30ccd387662eef1ee47187f41d5f6f9d1259f89db88eacaad358fec47 |
|
MD5 | c8e9375ce31d556a1e9d941a7711280a |
|
BLAKE2b-256 | dc87ba2792faded5e9f7aec4a2431fda35eb2c8df78a47d984ebf781c2ef2843 |
Close
Hashes for scrapysplashwrapper-1.6.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9e0535a48095e7762c4cc54226e00e41d6a483e64dc6908988fb1622704f5207 |
|
MD5 | 3f3c8c2fc8c71914c3271bb1b4b5f845 |
|
BLAKE2b-256 | ecf446a66815d538b7f9a462ee6b675334b9e796bf629986eb2004bd61f7db0a |