Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.7.0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c64f9df7127b2d40aa2c4ed45bab9faaa031a3a1770b426fd3c7397c05f3badc |
|
| MD5 | 77eca82939b743cfd75d8359baa429ae |
|
| BLAKE2b-256 | 25b06af57b1e21ba6173082d9cf930f6ac77787e7cf57ca6d8630a26d82d7abd |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.7.0-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 15ed46346f5d490b6336b84cdbc94c4a245668f7aa7d0a0d891e561a4de0ee58 |
|
| MD5 | 016ab6bc7aada5916ebf9635aae31f17 |
|
| BLAKE2b-256 | 6d3f086472319e466d708505debefc583401a6865f36d740fadab5722bc651a6 |