Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.4.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 5576516ac4be5605f95ac0de152e6d5971dbfd642d84457a2b69772dd1a4f1b4 |
|
| MD5 | ed1805a436f7fb1cd2d303e82bd2d7be |
|
| BLAKE2b-256 | 3f847ddec17dc47f4423d31a3ba53262fa966d2fe0cab0d0ac44b0ecb6464993 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.4-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 23e8332a346102473a132764a5cdbea21a7d4e5efb38198a4659f04e817d22a3 |
|
| MD5 | 462556d0739f62274c1d1020180d31e8 |
|
| BLAKE2b-256 | 258f774a002539d702223c5a5a849fa9b5cb0bb95e5e560df878c4a0ab622e0c |
