Skip to main content

AMQP based worker/master pattern framework

Project description

AMQP based RPC library for Tornado

Use cases

  • Task queue

  • Load balancing for different CPU-bound HTTP handlers

  • … other systems which involve RPC

Installation

pip install crew

Example

See the full example.

Usage

For example create your first app, and save as master.py:

import tornado.ioloop
import tornado.gen
import tornado.web
import tornado.log
import jaon
import tornado.options
from crew import TimeoutError, ExpirationError
from crew.master.tornado import Client

class MainHandler(tornado.web.RequestHandler):

    @tornado.gen.coroutine
    def get(self):
        resp = yield self.settings['crew'].call(None, priority=100)
        self.write(json.dumps(resp)

cl = Client()
application = tornado.web.Application(
    [
        (r"/", MainHandler),
    ],
    crew=cl,
    autoreload=True,
    debug=True,
)

if __name__ == "__main__":
    cl.connect()
    tornado.options.parse_command_line()
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

And create your first task, and save as worker.py:

# encoding: utf-8
from crew.worker import run, context, Task

@Task('test')
def long_task(req):
    context.settings.counter += 1
    return {"counter": context.settings.counter}

run(
    counter=0
)

After that run it:

$ python master.py &
$ python worker.py &
$ wait

Try to test it:

$ curl http://localhost:8888/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crew-0.8.8.tar.gz (10.4 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page