Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.6.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | eb80cfdf87823b9e5507e097f0e0b02bc0a9786bbdedce42be9d0339d8d84032 |
|
MD5 | 62170ad80253dd7a3009d3757110da62 |
|
BLAKE2b-256 | f185cbd8d9485509bfe6906bf1d1a9391d7d256757e3c098849ccf525d3b0732 |
Close
Hashes for scrapysplashwrapper-1.6.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c9c190052c14d978f25138210a9ad227d563636a2c7e6bd8bcca5b43b38e5228 |
|
MD5 | 812ac1c4af181b64b1b096529ae139aa |
|
BLAKE2b-256 | 27be3cee687b39ab137f00532229536ece31a3adb8f5a31a6e3e2e7c0f90173d |