Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Scrapy worker based on buildout with JSON-RPC 2.0 API

Project Description

This package provides a daemon which is able to run buildout based scrapy spider packages. (see s01.demo). The daemon processes pending crawl jobs stored in mongodb. The daemon also provides a WSGI server providing a JSON-RPC 2.0 API. The JSON-RPC proxy from the s01.client package can be used for install packages and add new crawl jobs to the mongodb which is the queue.

Changes

0.5.0 (2012-11-18)

  • initial implementation
Release History

Release History

This version
History Node

0.5.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
s01.worker-0.5.0.zip (73.1 kB) Copy SHA256 Checksum SHA256 Source Nov 18, 2012

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting