Skip to main content

Scrapy entrypoint for Scrapinghub job runner

Project description

Scrapy entrypoint for Scrapinghub job runner.

The package implements a base wrapper layer to extract job data from environment, parse/prepare it properly and execute job using Scrapy or custom executor.

Features

  • parsing job data from environment
  • processing job args and settings
  • running a job with Scrapy
  • collecting stats
  • advanced logging & error handling
  • full hubstorage support
  • custom scripts support

Install

pip install scrapinghub-entrypoint-scrapy

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapinghub-entrypoint-scrapy-0.12.1.tar.gz (26.6 kB view hashes)

Uploaded source

Built Distribution

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page