Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
- parsing job data from environment
- processing job args and settings
- running a job with Scrapy
- collecting stats
- advanced logging & error handling
- full hubstorage support
- custom scripts support
Requirements
- Python 2/3
- Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
| Filename, size | File type | Python version | Upload date | Hashes |
|---|---|---|---|---|
| Filename, size scrapinghub_entrypoint_scrapy-0.8.11-py2-none-any.whl (29.4 kB) | File type Wheel | Python version py2 | Upload date | Hashes View hashes |
| Filename, size scrapinghub-entrypoint-scrapy-0.8.11.tar.gz (22.0 kB) | File type Source | Python version None | Upload date | Hashes View hashes |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.11-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 32b0dd16177a0ad332595d923aee8896616cd5cd69ad7a6293b2528c44b59eca |
|
| MD5 | ad77887a8afa463fe28bd7c30c067733 |
|
| BLAKE2-256 | 48aeb85fa38d71e961286aaeaa0bcd0b1eeaf591f7cda1ef95ad1d9950e9b6ae |
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.11.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 76270983fdad0241902fcb0fc5208480efd721514f3c43e26c74e62582bd4c03 |
|
| MD5 | 995a2bcf83a78ea9b62b71c97908457c |
|
| BLAKE2-256 | 7b07c838cd2b9ad6909047633dc68a7e807b5cb8acd2bca2f08edb022b3b1cbc |
