Skip to main content

Scrapy splash wrapper as a standalone library.

Project description

Documentation Status

ScrapySplashWrapper

A wrapper that uses scrappy and splash to crawl a website.

Usage

Warning: it requires a splash instance (docker is recommendended).

usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
               [--debug]

Crawl a URL.

optional arguments:
  -h, --help            show this help message and exit
  -s SPLASH, --splash SPLASH
                        Splash URL to use for crawling.
  -u URL, --url URL     URL to crawl
  -d DEPTH, --depth DEPTH
                        Depth of the crawl.
  -o OUTPUT, --output OUTPUT
                        Output directory
  -ua USERAGENT, --useragent USERAGENT
                        User-Agent to use for crawling
  --debug               Enable debug mode on scrapy/splash

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapysplashwrapper-1.11.0.tar.gz (9.7 kB view details)

Uploaded Source

Built Distribution

scrapysplashwrapper-1.11.0-py3-none-any.whl (11.4 kB view details)

Uploaded Python 3

File details

Details for the file scrapysplashwrapper-1.11.0.tar.gz.

File metadata

  • Download URL: scrapysplashwrapper-1.11.0.tar.gz
  • Upload date:
  • Size: 9.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.13 CPython/3.9.7 Linux/5.13.0-35-generic

File hashes

Hashes for scrapysplashwrapper-1.11.0.tar.gz
Algorithm Hash digest
SHA256 563475623e0c9f8d09249df20cd65ffff8136b17dbc39ecb6cfb804f7f0a835b
MD5 86b11d1e45d939aff4cb57e7128757d4
BLAKE2b-256 18ac183069b4edf74bdd860a07d807da94c3d63843afd837a494ef3f7dbfd9b6

See more details on using hashes here.

File details

Details for the file scrapysplashwrapper-1.11.0-py3-none-any.whl.

File metadata

File hashes

Hashes for scrapysplashwrapper-1.11.0-py3-none-any.whl
Algorithm Hash digest
SHA256 04653a97ca0f44afd1c65902bf7055f0c476c7356843c76bc3b4098a1acbf693
MD5 a57763bb119ea5c19f88c3084f41b455
BLAKE2b-256 be470426dfdd492c3676634997aea6e1fd4f7bd07a86ef0139f5bff533bfc5da

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page