Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.2.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2dc99037a6c72f9d796c6ba57e69abeaebbaca6a13cfca41d9ac79d66ef26c83 |
|
MD5 | e643cfd867e4653eba93a1bce13e3713 |
|
BLAKE2b-256 | b6ce173256113eb72fe2526719c00497ba49130fa3d2ab771acdd8b3f7c9362b |
Close
Hashes for scrapysplashwrapper-1.2.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 527fa816517ac260b029b7c045101493a5cce10b1664e2fc07d723ba26531d7e |
|
MD5 | 5a96237d4c31bd69031f2958d19bee36 |
|
BLAKE2b-256 | 4707a9e5fe7017b99140c73737993875c27fca17df9101d9e08f81e2963df218 |