Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.6.1.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b91ab8b6be471b81e7467db5e9a6165591b7b06deacdf7ed20375e3f572cc6e4 |
|
| MD5 | 27a6a7612ef95146eccddbcb7c7a0947 |
|
| BLAKE2b-256 | b9c72f30efa9d92320bca0ac59e6add71e129a8f233d4f069ac0ccbddc7334c3 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.6.1-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | ce0b136eec77d488ea2f74f94c85bd1250125ba3b4ffa7bad729f8315a48c81c |
|
| MD5 | 01c7cce41ed934f8c332b9d354a2de04 |
|
| BLAKE2b-256 | 13d893adf07728748262866e58f01cca2cfef768515ed68f46a1397e2c9ba5a9 |
