Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.2.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d1185c760dde1bcda389223545ea8fe16bd9308e52c9baa52f654fb91c7a787f |
|
MD5 | 25bf78b9197224f9a7b91b58d8b0cf63 |
|
BLAKE2b-256 | c005e82460417d0b5dfaee0f5dbc5e1d23ac26bbb603de3b60e404d4d5c6a2e7 |
Close
Hashes for scrapysplashwrapper-1.2.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 11930ef076d0fdba66caa6045f8c9c142247cb4eb0ba0939e08306a5b4e879bf |
|
MD5 | cde48bbd4e330c6367ae9bc95c27f0c5 |
|
BLAKE2b-256 | 4f32a6c9ceb5e469fde15143954ae500679ca8dc3d13c27e4a761a43aae87990 |