Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1ac854f4c4e5a7a594d2e1a39d94330b67359420e16c7f1adc2a016579fcc16c |
|
MD5 | 74dca622c635579c14d00bb61b7876fa |
|
BLAKE2b-256 | 3e02c593e07f7bb22db9cc8ee672b92589d093f53d3999ce2f3ee12bb2ed26be |
Close
Hashes for scrapysplashwrapper-1.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 660275a5a6f899e09abf8b732e0724a280cab6b44cb3405c85a92e25b87dac6b |
|
MD5 | 7cd51b2a4095a3f98098b6d6a91f8194 |
|
BLAKE2b-256 | 8ec5ffb8f89687cdf92864b01486ac6e65618c9215d045799e374ec7d6453761 |