Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.12.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 9994f73ace9203bf1185c3f23740345443ac04cb01918ff9050d177137ab05ad |
|
| MD5 | 4bfb54fc7ec836597b18989c276beef8 |
|
| BLAKE2b-256 | 5c4af84d87eed1a6a8165822ec97d1260747f140d64f8c4b6149c3e2e052c791 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.12-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 9a4105a262a0b9bf872a18242e5b45fc3bdb83fc96812b827b749a81539a544f |
|
| MD5 | 568a170976a9426582a18b79320d0e66 |
|
| BLAKE2b-256 | 540968d7ab0e47cbc551f204d7594c9c11e201c515e1a68b73a041a5cafd4115 |
