Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.8.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 33f3a3ba4247024974b81d5fa3a0dddcbab24f3de06674c2687dfc6ffb711541 |
|
MD5 | 95090e232e1d0e49856af08d66f9f363 |
|
BLAKE2b-256 | 9bfa2667b29a1613eaef2128a9d873c04ff0731d363825e6caf6b8a9c9be41af |
Close
Hashes for scrapysplashwrapper-1.8.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 053e01e9954087547d21cd4a36d8b4330627c146c1fa13ec846d691adc8826df |
|
MD5 | 383a3931d6d668c887b7c7e92a8c5259 |
|
BLAKE2b-256 | abcd4ba0543d364284b7046826d7f1e27943190a5daf6138ef7b4d0a21db6ece |