Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.9.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c348cc71730f981105fc09752869fc84debbf55e0a4d6f4c34a8aed88df2edd4 |
|
MD5 | 2b19ad5810fec3ca75b3437b6b9b13b8 |
|
BLAKE2b-256 | 72e4238c0e0e0efe39bac3080d211552f10b03e86f133b276a93f3f76ccbc342 |
Close
Hashes for scrapysplashwrapper-1.9.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5d8b0de7cd07e037223b706e967af3c069613d795d94511616b8f8abee798184 |
|
MD5 | e7242e7ee3dd8a38260276650203a796 |
|
BLAKE2b-256 | aeb694a90410350af753aceec49515fb38c1057c71ba0b066424e63be1b4e103 |