Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.9.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 95fdfb934f929e27f2e6f3f6d040075611c09f34ca08a34e0870389ef59694c2 |
|
MD5 | 6803f287627ebed408577b2d7f9d66f2 |
|
BLAKE2b-256 | c826e30d5f425079f84a2e394e34e88cd73a4ad3949ea69d16d7325b3a57f3cb |
Close
Hashes for scrapysplashwrapper-1.9.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 27f7ece8dadaaaa8bdc71da6a9a1d6b49630d3315e629c2fd6f69844aafa1988 |
|
MD5 | 4cc711a747e27bb9527272bea9da5a6a |
|
BLAKE2b-256 | d85aa46b3c01cd0aaff7ec2bc7d7e5bcddb6abf13ee8bb2ae0b2e02ffa41b174 |