Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.6.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 594b7f4990a86379ad972bdc6d81d066f6d275390d00361ec71f42fce5b770d0 |
|
MD5 | 91f56b3f3bdb85f6f92fa3cd4b966ce0 |
|
BLAKE2b-256 | b98b5659597bfe297f5cedb1bead614ff47de99a2aab5762c8d73894202a188c |
Close
Hashes for scrapysplashwrapper-1.6.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 05b0bc668238349e5a251d544ffa112039e88b165f9490262e27f894db9117ce |
|
MD5 | f2660fb6a937db9a028a68d91a2e4157 |
|
BLAKE2b-256 | cef6970ed3117749d1c5ae8a33a194ba18c31b013097de7b310a42ac1ab08f29 |