Scrapy worker based on buildout with JSON-RPC 2.0 API
Project description
This package provides a daemon which is able to run buildout based scrapy spider packages. (see s01.demo). The daemon processes pending crawl jobs stored in mongodb. The daemon also provides a WSGI server providing a JSON-RPC 2.0 API. The JSON-RPC proxy from the s01.client package can be used for install packages and add new crawl jobs to the mongodb which is the queue.
Changes
0.5.0 (2012-11-18)
initial implementation
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
s01.worker-0.5.0.zip
(73.1 kB
view hashes)
