ScrapySplashWrapper

A wrapper that uses scrappy and splash to crawl a website.

Usage

Warning: it requires a splash instance (docker is recommendended).

usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
               [--debug]

Crawl a URL.

optional arguments:
  -h, --help            show this help message and exit
  -s SPLASH, --splash SPLASH
                        Splash URL to use for crawling.
  -u URL, --url URL     URL to crawl
  -d DEPTH, --depth DEPTH
                        Depth of the crawl.
  -o OUTPUT, --output OUTPUT
                        Output directory
  -ua USERAGENT, --useragent USERAGENT
                        User-Agent to use for crawling
  --debug               Enable debug mode on scrapy/splash

Metadata

Release files for scrapysplashwrapper 1.11.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for scrapysplashwrapper 1.11.0
File	Size	Uploaded
scrapysplashwrapper-1.11.0.tar.gz	9.7 kB	Mar 31, 2022	Details

Built distribution (wheel)

Table of built distributions (wheels) for scrapysplashwrapper 1.11.0
File	Interpreter	ABI	Platform	Reset
scrapysplashwrapper-1.11.0-py3-none-any.whl 11.4 kB Mar 31, 2022	Python 3	none	any	Details

Total release size: 21.2 kB

Release files / scrapysplashwrapper-1.11.0.tar.gz

Download URL	scrapysplashwrapper-1.11.0.tar.gz
Size	9.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`563475623e0c9f8d09249df20cd65ffff8136b17dbc39ecb6cfb804f7f0a835b`
BLAKE2b-256 checksum How to use checksums	`18ac183069b4edf74bdd860a07d807da94c3d63843afd837a494ef3f7dbfd9b6`
Upload date	Mar 31, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.1.13 CPython/3.9.7 Linux/5.13.0-35-generic`

Release files / scrapysplashwrapper-1.11.0-py3-none-any.whl

Download URL	scrapysplashwrapper-1.11.0-py3-none-any.whl
Size	11.4 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`04653a97ca0f44afd1c65902bf7055f0c476c7356843c76bc3b4098a1acbf693`
BLAKE2b-256 checksum How to use checksums	`be470426dfdd492c3676634997aea6e1fd4f7bd07a86ef0139f5bff533bfc5da`
Upload date	Mar 31, 2022
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.1.13 CPython/3.9.7 Linux/5.13.0-35-generic`