Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.6.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d5e5b52efe61a6685be6e7e817829f81156ec1f8982fdb463b80891220d918ce |
|
MD5 | 66c0e7cbcfb12e2b5c478e92b18c4ec1 |
|
BLAKE2b-256 | 531a5bf6c10a8508a71e114c1ff98e743b95d8faf9c36d7ddf54112321c6dfdf |
Close
Hashes for scrapysplashwrapper-1.6.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f1ac81d4d78b9cb00149dbe230bdb4b36d2bc55c4958d8c8e966164d6fe62808 |
|
MD5 | f0e82e17361e228c305f937fc644a646 |
|
BLAKE2b-256 | 21bd9c72b5794565faf3cca5ed6c1a5ea08a5c102c1a35ff9df627e660b6bc71 |