Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.2.7.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2b1f8cfd1f31d834054c1c38de5f01453db51dfc0fdacd739b7d4156cd55b44c |
|
MD5 | b4712fd024d35185175d0572b30a9cf2 |
|
BLAKE2b-256 | 4c6a4fb91f0364dbe5c421c67a5ecc9947bb74fe1a22a7e63684475f1675df49 |
Close
Hashes for scrapysplashwrapper-1.2.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f147ec57cbb86e9f5fd064ab606b1af35f8cd3c51f9099784b2ed62daf288a4 |
|
MD5 | 9e4c2e1904173605a708f9341786bdef |
|
BLAKE2b-256 | 7b78fbd3eb52bf0990e22f6488e3c73419c28bf41c6968eeb7cbc15c9a1e7075 |