Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3ecb4455d8949c4fb7e2ba868efd71fcd4ae746be28c5eaa1f8a884312c544ec |
|
MD5 | eab36a754868b70887249f22bab335a5 |
|
BLAKE2b-256 | 9e7e9cfaa9e0a60aba2c12fcd6b826c9c471d02b44f8d176c5e9114e2651b1cd |
Close
Hashes for scrapysplashwrapper-1.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 02460ec8714206045aa64c5586765fb8b5ba57e4aecfb3c08bd36b2c10b6b546 |
|
MD5 | b9df010abfeb2c8cc40f2329fc109778 |
|
BLAKE2b-256 | a2c78480984c46d6c2e9eebbd815240d856cd371ff89a70c36faef3712f0a773 |