Skip to main content

Mutex for Celery tasks, which can be refined.

Project description

Celey Mutex
===========

Celery Mutex is a mutex for Celery Tasks, optionally refined based on provided
keys. This mutex prevents execution of concurrent tasks, as opposed to delaying
execution.


Installation
------------

Simply run `pip install celery_mutex`


Requirements
------------

Celery Mutex relies on [ZooKeeper](http://zookeeper.apache.org/).


Usage
-----

Celery Mutex takes advantage of Abstract Tasks for Celery. To utilize this
abstract task, add it as a base for the task:

import celery_mutex

@app.task(base=celery_mutex.MutexTask)
def my_task(a, b):
return a + b

A new configuration variable is required in order to let Celery Mutex know
about your ZooKeeper servers:

ZOOKEEPER_HOSTS = 'localhost:2181'

This is a comma-separated list of hosts to connect to.

By default a mutex times out after one hour. This can be changed globally by
setting `MUTEX_TIMEOUT` or per-task by setting `mutex_timeout` on the task. For
both the value is an integer for the number of seconds to set the time out.

A second configuration option allows you to refine the mutex for a given task.
By default, Celery Mutex only allows one instance of a task at a time. However,
there may be a need to further refine what is controlled by the mutex. This can
be done by setting `mutex_keys` on the task. The value is a list of keys that
are to be used for determining exclusivity.

Using our above example, adding the two optional parameters would yield:

import celery_mutex

@app.task(base=celery_mutex.MutexTask, mutex_timeout=30, mutex_keys=('a',))
def my_task(a, b):
return a + b

This would cause the mutex to only prevent execution for tasks that share the
same value for "a".

Project details


Release history Release notifications

This version
History Node

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
celery_mutex-0.1.0-py27-none-any.whl (5.5 kB) Copy SHA256 hash SHA256 Wheel any Mar 27, 2014
celery_mutex-0.1.0.tar.gz (3.5 kB) Copy SHA256 hash SHA256 Source None Mar 27, 2014

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page