Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.2.8.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6505c3fa51fd6e389155daf9b3c5f176a8d92ecebc64a260184d84d3fcf9e0e3 |
|
MD5 | 4cfb2d69b102e39734d177c0809e994d |
|
BLAKE2b-256 | 97054e6147e7b069838c955a2e7faf696d1f2d8ff37f49841e5918812b834aa0 |
Close
Hashes for scrapysplashwrapper-1.2.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0cd0df258c018f6dd619ffd1d78f2a48e5670304937110e68491608cf1f4ffa5 |
|
MD5 | 50ca54e59999db303bfe0247d6b40f4a |
|
BLAKE2b-256 | b033b72beb41d02265422848d2fa1fb22298afdec9e9b28377dbabe765d6899c |