Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
1 maintainer
Metadata
1 maintainer
Release files for scrapinghub-entrypoint-scrapy 0.8.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| scrapinghub-entrypoint-scrapy-0.8.9.tar.gz | 21.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| scrapinghub_entrypoint_scrapy-0.8.9-py2-none-any.whl | Python 2 | none | any | Details |
Total release size: 50.0 kB
Release files / scrapinghub-entrypoint-scrapy-0.8.9.tar.gz
| Download URL | scrapinghub-entrypoint-scrapy-0.8.9.tar.gz |
|---|---|
| Size | 21.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
eb475c12632863e7de8f0eefc7a059fc2dfa4851e7111432cd395a805e7b2acf
|
|
BLAKE2b-256 checksum How to use checksums |
882e005f9730ea51a069d7cfcc45de3539f1e369c8c76a76b3a52c68354c916d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Release files / scrapinghub_entrypoint_scrapy-0.8.9-py2-none-any.whl
| Download URL | scrapinghub_entrypoint_scrapy-0.8.9-py2-none-any.whl |
|---|---|
| Size | 29.0 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
f7c83e8f61dde8eb483252235eabd452ccabb670eefb3a2494a64cc3b14be3f4
|
|
BLAKE2b-256 checksum How to use checksums |
565c07149a548401b8b6a6fbb50b648c73efca78d2a0926e4c457a039dde363e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
