Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.5.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5892cb842e1ad789943edf7d79d43795b3c20946b2292a6e91ed4dff6df709b8 |
|
MD5 | 2fa23a9340b47e82e4323843e5a355cf |
|
BLAKE2b-256 | a0df3cfdbfe330b9c4c12e5d1f6a1cc81375ba661f2cb42a197897b8c71ebc65 |
Close
Hashes for scrapysplashwrapper-1.5.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1d4e0216e3e1f6e15c721d0cf3eaa673fb54396550ece36cb9495fc384a27662 |
|
MD5 | de885a72b4ce5ff068ac781aea446f99 |
|
BLAKE2b-256 | 0677db6f3f5a2cc3073c8c98faaeaa98443d5b3694aaee18f96ae161d4305c6b |