Qamasu is Job Queue system that respects TheSchwartz.
Project description
Qamasu is JobQueue system that respects TheSchwartz.
Suited to load leveling.
Implemented using optimistic lock.
Requirements
Python>=2.6
Django>=1.0
Python3 needs Django1.5
Usage
Set Qamasu up!
Qamasu is a Django application.
You need add qamasu to your or new django project’s INSTALLED_APPS.
And //manage.py syncdb//.
Write your worker.
Define GRAB_FOR in seconds that is max time worker grabbed for a work.
Define def work_safely(manager, job): that is a work you need.
See sample worker in workers directory for detail.
Registration
You need add worker to abilities.
register_func insert fuction record into database table if not exist.
>>> from qamasu import Qamasu >>> qamasu = Qamasu([]) >>> qamasu.register_func('workers.random_wait')
Queue!
Once fuction is registered to qamasu, you can enqueue jobs.
Add hundreds Queues.:
>>> from qamasu import Qamasu >>> from random import uniform >>> qamasu = Qamasu(['workers.random_wait',]) >>> for x in xrange(1,500): arg = dict(random_number=uniform(1,5)) qamasu.enqueue('workers.random_wait', arg)
Add a highest-priority queue.:
>>> qamasu.enqueue('workers.random_wait', dict(random_number=uniform(1,5)), priority=1)
Work! Work! Work!
Process enqueued job.
Instantiate Qamasu with availabilities.
call work method. infinite loop inside this method.
work method process queue as FIFO.
>>> from qamasu import Qamasu >>> qamasu = Qamasu(['workers.random_wait',]) >>> qamasu.work()
Use work_prioritizing method if you tend to process job respects to priority.
>>> from qamasu import Qamasu >>> qamasu = Qamasu(['workers.random_wait',]) >>> qamasu.work_prioritizing()
Caution!
For MySQL backend
You must set worker’s transaction isolation level to read commited before working qamasu when you use InnoDB.
>>> from django.db import connection >>> from qamasu import Qamasu >>> connection.cursor().execute('set session transaction isolation level read committed') >>> qamasu = Qamasu(['workers.random_wait',]) >>> qamasu.work()
Or you have to set transaction isolation level read committed. It’s global settings and dangerous.
[mysqld] transaction-isolation=Read-Committed
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.