Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.10.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b625de144cffacb00a6cba231326099bd36f8882a7b95f734869c8ca6e15c8bf |
|
MD5 | 03b9f385dccd2f08998b57c8dc8a2be5 |
|
BLAKE2b-256 | cf1f79189bb6b6d891753bb66756cde539239e5f3273b05644511531ddd72702 |
Close
Hashes for scrapysplashwrapper-1.10.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c43c373a292669c484c0686df0aa3553b7569a06e7a7d8b62f38211bac101d9d |
|
MD5 | 1ec2eee39f6574f887f399b1dccfe19d |
|
BLAKE2b-256 | 4e7be04db26eb775df4d0122930ca75d767f34d40b8b72b3fd363d2ec4d64edf |