Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
- parsing job data from environment
- processing job args and settings
- running a job with Scrapy
- collecting stats
- advanced logging & error handling
- full hubstorage support
- custom scripts support
Requirements
- Python 2/3
- Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
| Filename, size | File type | Python version | Upload date | Hashes |
|---|---|---|---|---|
| Filename, size scrapinghub_entrypoint_scrapy-0.7.0-py2-none-any.whl (12.8 kB) | File type Wheel | Python version py2 | Upload date | Hashes View hashes |
| Filename, size scrapinghub-entrypoint-scrapy-0.7.0.tar.gz (9.1 kB) | File type Source | Python version None | Upload date | Hashes View hashes |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.7.0-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 15ed46346f5d490b6336b84cdbc94c4a245668f7aa7d0a0d891e561a4de0ee58 |
|
| MD5 | 016ab6bc7aada5916ebf9635aae31f17 |
|
| BLAKE2-256 | 6d3f086472319e466d708505debefc583401a6865f36d740fadab5722bc651a6 |
Close
Hashes for scrapinghub-entrypoint-scrapy-0.7.0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c64f9df7127b2d40aa2c4ed45bab9faaa031a3a1770b426fd3c7397c05f3badc |
|
| MD5 | 77eca82939b743cfd75d8359baa429ae |
|
| BLAKE2-256 | 25b06af57b1e21ba6173082d9cf930f6ac77787e7cf57ca6d8630a26d82d7abd |
