Scrapy splash wrapper as a standalone library.

These details have not been verified by PyPI

Project links

Project description

ScrapySplashWrapper

A wrapper that uses scrappy and splash to crawl a website.

Usage

Warning: it requires a splash instance (docker is recommendended).

usage: scraper [-h] [-s SPLASH] -u URL [-d DEPTH] [-o OUTPUT] [-ua USERAGENT]
               [--debug]

Crawl a URL.

optional arguments:
  -h, --help            show this help message and exit
  -s SPLASH, --splash SPLASH
                        Splash URL to use for crawling.
  -u URL, --url URL     URL to crawl
  -d DEPTH, --depth DEPTH
                        Depth of the crawl.
  -o OUTPUT, --output OUTPUT
                        Output directory
  -ua USERAGENT, --useragent USERAGENT
                        User-Agent to use for crawling
  --debug               Enable debug mode on scrapy/splash

Project details

These details have not been verified by PyPI

Project links

Release history Release notifications | RSS feed

This version

1.11.0

Mar 31, 2022

1.10.1

Dec 13, 2021

1.10.0

Dec 3, 2021

1.9.3

Nov 23, 2021

1.9.2

Oct 14, 2021

1.9.1

Oct 7, 2021

1.9.0

Sep 28, 2021

1.8.1

Sep 11, 2021

1.8.0

Aug 30, 2021

1.7.0

Jul 21, 2021

1.6.3

Jun 29, 2021

1.6.2

Jun 29, 2021

1.6.1

Jun 2, 2021

1.6.0

May 21, 2021

1.5.1

May 12, 2021

1.5.0

Apr 2, 2021

1.4.0

Feb 9, 2021

1.3

Dec 23, 2020

1.2.8

Nov 10, 2020

1.2.7

Oct 29, 2020

1.2.6

Oct 12, 2020

1.2.5

Oct 3, 2020

1.2.4

Sep 15, 2020

1.2.3

Sep 8, 2020

1.2.2

Jul 23, 2020

1.2.1

Jul 20, 2020

1.2

Jul 20, 2020

1.1.2

Jul 7, 2020

1.1.1

Jul 3, 2020

1.1

Jun 10, 2020

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapysplashwrapper-1.11.0.tar.gz (9.7 kB view hashes)

Uploaded Mar 31, 2022 Source

Built Distribution

scrapysplashwrapper-1.11.0-py3-none-any.whl (11.4 kB view hashes)

Uploaded Mar 31, 2022 Python 3

Hashes for scrapysplashwrapper-1.11.0.tar.gz

Hashes for scrapysplashwrapper-1.11.0.tar.gz
Algorithm	Hash digest
SHA256	`563475623e0c9f8d09249df20cd65ffff8136b17dbc39ecb6cfb804f7f0a835b`
MD5	`86b11d1e45d939aff4cb57e7128757d4`
BLAKE2b-256	`18ac183069b4edf74bdd860a07d807da94c3d63843afd837a494ef3f7dbfd9b6`

Hashes for scrapysplashwrapper-1.11.0-py3-none-any.whl

Hashes for scrapysplashwrapper-1.11.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`04653a97ca0f44afd1c65902bf7055f0c476c7356843c76bc3b4098a1acbf693`
MD5	`a57763bb119ea5c19f88c3084f41b455`
BLAKE2b-256	`be470426dfdd492c3676634997aea6e1fd4f7bd07a86ef0139f5bff533bfc5da`