Skip to main content

Qamasu is JobQueue system that respects TheSchwartz.

Suited to load leveling.

Implemented using optimistic lock.

Requirements

  • Python>=2.6

  • Django>=1.0

Python3 needs Django1.5

Usage

Set Qamasu up!

Qamasu is a Django application.

You need add qamasu to your or new django project’s INSTALLED_APPS.

And //manage.py syncdb//.

Write your worker.

Define GRAB_FOR in seconds that is max time worker grabbed for a work.

Define def work_safely(manager, job): that is a work you need.

See sample worker in workers directory for detail.

Registration

You need add worker to abilities.

register_func insert fuction record into database table if not exist.

>>> from qamasu import Qamasu
>>> qamasu = Qamasu([])
>>> qamasu.register_func('workers.random_wait')

Queue!

Once fuction is registered to qamasu, you can enqueue jobs.

Add hundreds Queues.:

>>> from qamasu import Qamasu
>>> from random import uniform
>>> qamasu = Qamasu(['workers.random_wait',])
>>> for x in xrange(1,500):
      arg = dict(random_number=uniform(1,5))
      qamasu.enqueue('workers.random_wait', arg)

Add a highest-priority queue.:

>>> qamasu.enqueue('workers.random_wait', dict(random_number=uniform(1,5)), priority=1)

Work! Work! Work!

Process enqueued job.

  1. Instantiate Qamasu with availabilities.

  2. call work method. infinite loop inside this method.

work method process queue as FIFO.

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

Use work_prioritizing method if you tend to process job respects to priority.

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work_prioritizing()

Caution!

For MySQL backend

You must set worker’s transaction isolation level to read commited before working qamasu when you use InnoDB.

>>> from django.db import connection
>>> from qamasu import Qamasu
>>> connection.cursor().execute('set session transaction isolation level read committed')
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

Or you have to set transaction isolation level read committed. It’s global settings and dangerous.

[mysqld]
transaction-isolation=Read-Committed

Release files for qamasu 0.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for qamasu 0.5
File Size Uploaded
qamasu-0.5.tar.gz 6.5 kB Details

Release files / qamasu-0.5.tar.gz

Download URL qamasu-0.5.tar.gz
Size 6.5 kB
Tags Source
SHA-256 checksum
How to use checksums
fe2910f2b16c2933fbb8b6344b86a3bb8b0de1620b3fe13e47bcccd1ba728d0f
BLAKE2b-256 checksum
How to use checksums
87d28a1dbf840411f76e250a4f1f529d4afbdcd8f4182aac817ff9ceab1f9280
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release history Release notifications | RSS feed

This release

0.5 This release

1 release file

0.4

1 release file

0.3

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page