Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Scrapy worker based on buildout with JSON-RPC 2.0 API

Project description

This package provides a daemon which is able to run buildout based scrapy spider packages. (see s01.demo). The daemon processes pending crawl jobs stored in mongodb. The daemon also provides a WSGI server providing a JSON-RPC 2.0 API. The JSON-RPC proxy from the s01.client package can be used for install packages and add new crawl jobs to the mongodb which is the queue.

Changes

0.5.0 (2012-11-18)

  • initial implementation

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for s01.worker, version 0.5.0
Filename, size File type Python version Upload date Hashes
Filename, size s01.worker-0.5.0.zip (73.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page