Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file scrapinghub-entrypoint-scrapy-0.8.9.tar.gz.
File metadata
- Download URL: scrapinghub-entrypoint-scrapy-0.8.9.tar.gz
- Upload date:
- Size: 21.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
eb475c12632863e7de8f0eefc7a059fc2dfa4851e7111432cd395a805e7b2acf
|
|
| MD5 |
1d4eb3e1920afe800608a64d28e4a32b
|
|
| BLAKE2b-256 |
882e005f9730ea51a069d7cfcc45de3539f1e369c8c76a76b3a52c68354c916d
|
File details
Details for the file scrapinghub_entrypoint_scrapy-0.8.9-py2-none-any.whl.
File metadata
- Download URL: scrapinghub_entrypoint_scrapy-0.8.9-py2-none-any.whl
- Upload date:
- Size: 29.0 kB
- Tags: Python 2
- Uploaded using Trusted Publishing? No
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f7c83e8f61dde8eb483252235eabd452ccabb670eefb3a2494a64cc3b14be3f4
|
|
| MD5 |
03dc4a432cb4f27fc78b46e85230ee4a
|
|
| BLAKE2b-256 |
565c07149a548401b8b6a6fbb50b648c73efca78d2a0926e4c457a039dde363e
|
