Scrapy entrypoint for Scrapinghub job runner
Project description
Scrapy entrypoint for Scrapinghub job runner.
The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.
Features
parsing job data from environment
processing job args and settings
running a job with Scrapy
collecting stats
advanced logging & error handling
full hubstorage support
custom scripts support
Requirements
Python 2/3
Scrapy, python-hubstorage (check requirements.txt for details)
Install
The quick way:
pip install scrapinghub-entrypoint-scrapy
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapinghub-entrypoint-scrapy-0.8.6.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 078c81ee2631c8028e1dca3fffe9fd4c270dea23f8baacb72b558b581a17b7b5 |
|
| MD5 | 9fba34970fc45fbdc236785871e80e4f |
|
| BLAKE2b-256 | 551bab33aee68ba262e4f2aa27954f4bdea46727ee9806e1ab573a3f021ddc23 |
Close
Hashes for scrapinghub_entrypoint_scrapy-0.8.6-py2-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c780ad45ba801d6b0b477971c10b5d239a012066e9bbe5feaabf9132f66876c5 |
|
| MD5 | 941d89fcf53969ea72bd1cf3930bf70f |
|
| BLAKE2b-256 | 169b54609a55326db50962b6952d8459bf12bfa031cabf8db88d2428bf573b00 |
