Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.9.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3edb2860a35c4cf74e13c377130ebda777498e7f576e9b6aca2dd7b268d85934 |
|
MD5 | c23a6f8912a4181d813d5c0d5b4676b0 |
|
BLAKE2b-256 | 40a21bd86b3b1f1311f6a7503b6e3668476c661549e2c99cc6a0d6e130ec2542 |
Close
Hashes for scrapysplashwrapper-1.9.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ca63c67c0892ac1ab1a9db5c3e83711b7683306571a08fbfba43d9b71d55f826 |
|
MD5 | 226cabe2e05a762ed10a9e8a4b71256c |
|
BLAKE2b-256 | 3ed8282e4fed5fcb3967c0e36be9953d6d0c0525bd1a1071685ab90ac2d3a7a1 |