Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.9.0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 60853fc38506341cf8ace4cc37724dc9c4cf8ed8ffc4a62ef817d045de5868a2 |
|
| MD5 | ef40fc1a19d32933288f5f79678bba53 |
|
| BLAKE2b-256 | 3b7cd836635e70a993ef2a46f6b04b9ccea6f27018df41f4e05b41d080bd8583 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.9.0-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 029743394c65a6047f742ed542a84f87a2f2b599deab0d2af3358f7dd0e68171 |
|
| MD5 | 33ae242682b015cdedc5212899745afb |
|
| BLAKE2b-256 | 72e28e131872d884a625690c33b32d5267c75b17f900475602dbd131087605c7 |
