Scrapy splash wrapper as a standalone library.
Project description
ScrapySplashWrapper
A wrapper that uses scrappy and splash to crawl a website.
Usage
Warning: it requires a splash instance (docker is recommendended).
usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
[--debug]
Crawl a URL.
optional arguments:
-h, --help show this help message and exit
-s SPLASH, --splash SPLASH
Splash URL to use for crawling.
-u URL, --url URL URL to crawl
-d DEPTH, --depth DEPTH
Depth of the crawl.
-o OUTPUT, --output OUTPUT
Output directory
-ua USERAGENT, --useragent USERAGENT
User-Agent to use for crawling
--debug Enable debug mode on scrapy/splash
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapysplashwrapper-1.2.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4085a1f583ecb041edea8663f450d82782d574f46f951cbcc755267a235b08ef |
|
MD5 | 0a8647bbf7a300a5e717920e962336df |
|
BLAKE2b-256 | 488064b6d0b80b7fd7d0ad82424aac256a412fb30f402ea590a5abf0e233cd08 |
Close
Hashes for scrapysplashwrapper-1.2.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | de03a32f1e8a23daf6525651908f520412c4e1bfa489652306c4f39d374801f5 |
|
MD5 | d580d753f0df003eaac26746df44dc83 |
|
BLAKE2b-256 | 6fac0471283a4cfc213d0ba0f00828c99f5d00da10711cdfc35d410d137b514c |