Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.10.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 82394d0317e4d19654ee1c3382a200ab1202b574226612f9cc3e6250abd38532 |
|
MD5 | 91cc43e57e6b35c52cdf369e7daef437 |
|
BLAKE2b-256 | 0d4823589dad5b1ec33c43152cfa7c17eed9bcac42aea1ba4036f556816a23a8 |
Close
Hashes for scrapysplashwrapper-1.10.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b0ff416733d582b64c8dd1b19ff850d349b7c5911a15da78b77c9da6f756f4f7 |
|
MD5 | d801cd3d23a21b274384d52c17041af6 |
|
BLAKE2b-256 | a7f47098ec0f3f28148ccd517ea07435ca662b6e9c1945495920fb5c6fd109b9 |