Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.5.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | d783238b28232087d485fa72f5f60638e18057f05705ed88cb93dfa0f74d8570 |
|
| MD5 | 1a48c82120e337ce1be44e6bc2c12f2a |
|
| BLAKE2b-256 | 36973373b688b8750d5859882b26b6cdb0d71718294f8d1a800a9c06b67ab4a0 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.5-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 814e961d7214f1de325bf4cc1b9cd9ec54127678fa75a9950a9953111288d14d |
|
| MD5 | b9d23439d829511f2e597d09ce862817 |
|
| BLAKE2b-256 | 84ee2a35e95fad6086fdb7497ddd241d350f5a4fe0f72f3f0c3654a1a0b2795a |
