Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.3.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | fda729ee69be972144e7a9ab14fa55d97bcd65e6c4bad4fd340d528e37f1ea3e |
|
| MD5 | 95029e826e15ce630cf5dafa393fb7a6 |
|
| BLAKE2b-256 | be0f9433d0858847c14b9f72281a08d6a68b1ab6c02afe466b1b047249a5d8fe |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.3-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 0e032ec87b3acfc4dd045e646b2d225d8c5fd576d15ca3e1bd19e8fcda6a53a3 |
|
| MD5 | 206027f7ff0c79d590e80cbabe763326 |
|
| BLAKE2b-256 | 5eb6d8a28e356b74eacb8f47253aac229cc581704062b01e51aa521399240d17 |
