Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.5.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1c4ee6cbb47749d91e9e132a5a86de1dffa61c77e9b830b39d9dc95dadcd774b |
|
MD5 | 3dc6749e7e86301697850caecd364e87 |
|
BLAKE2b-256 | b5407427c8e2af277ccef00854314afea6d48e00ee29281a1e5d6f8b41928f0c |
Close
Hashes for scrapysplashwrapper-1.5.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f29715b30ac6acc01d64c67b1ea6214aca794fa88363c3602a039294bc65f96 |
|
MD5 | ef7110c23a608b4c3e014616a3ce7f5e |
|
BLAKE2b-256 | 7d2202a4b98476e23ee7e0d8f42e05834d2ed3c9c851bd14c691b824de3bc5d8 |