Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.2.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 8d216d4e5c37148396740c93b89e794a168cc9f31ac699117cce7b56e95be015 |
|
| MD5 | 449030076fdee36a6fea9d0a9cef1c2b |
|
| BLAKE2b-256 | e05c6f45b4979f913478fed8c4c865bbe1adf30e3922c69fa0c5f38d84fa7448 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.2-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 22349b82314b5430cba74f6fe24c97bfe66dd5cb5430c9a42ef0ed48e51c28d3 |
|
| MD5 | ba4b953f2d2445fd5ab543e71aa010a3 |
|
| BLAKE2b-256 | 396f8dd902caa8af35c80f8fa2a877694e98a006c5257004ba7212814a4a6799 |
