Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.9.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0621a02fffba2d9eab8e0bea3e986474b3d9edfcd5681f41a8162c8bba517490 |
|
MD5 | bc6ff3930b82a613045c687836b519a0 |
|
BLAKE2b-256 | ac5af08d723bf827f9c2261389ebc59757aac1971e3cce6d4a99ae0efe5e63fa |
Close
Hashes for scrapysplashwrapper-1.9.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ece1dbf0ec4c0b9fe34f34bc063ae27592cd89fe2d205c2c476514baf4b1c372 |
|
MD5 | 45de81839ce771c0eb82a202e5925f12 |
|
BLAKE2b-256 | 71070eb8f3964f3a9a8869f74eef54d3d2b3b8ed19e3b9cdde1dff75110994dd |