Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
- parsing job data from environment
- processing job args and settings
- running a job with Scrapy
- collecting stats
- advanced logging & error handling
- full hubstorage support
- custom scripts support
Requirements
- Python 2/3
- Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
| Filename, size | File type | Python version | Upload date | Hashes |
|---|---|---|---|---|
| Filename, size scrapinghub_entrypoint_scrapy-0.8.9-py2-none-any.whl (29.0 kB) | File type Wheel | Python version py2 | Upload date | Hashes View hashes |
| Filename, size scrapinghub-entrypoint-scrapy-0.8.9.tar.gz (21.0 kB) | File type Source | Python version None | Upload date | Hashes View hashes |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.9-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | f7c83e8f61dde8eb483252235eabd452ccabb670eefb3a2494a64cc3b14be3f4 |
|
| MD5 | 03dc4a432cb4f27fc78b46e85230ee4a |
|
| BLAKE2-256 | 565c07149a548401b8b6a6fbb50b648c73efca78d2a0926e4c457a039dde363e |
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.9.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | eb475c12632863e7de8f0eefc7a059fc2dfa4851e7111432cd395a805e7b2acf |
|
| MD5 | 1d4eb3e1920afe800608a64d28e4a32b |
|
| BLAKE2-256 | 882e005f9730ea51a069d7cfcc45de3539f1e369c8c76a76b3a52c68354c916d |
