django-async 0.3

Asynchronous task execution with proper database transaction management for Django

Django Async is an asynchronous execution queue for Django with proper database transaction management.

Building a database backed task queue is a fairly trivial thing, but getting the database transactions exactly right is no simple matter.

Using Django Async

Installation is very simple, just add the async application to your Django applications in settings.py.

To run a job asynchronously just use the schedule function:

from async import schedule
schedule('my.function', args=(1, 2, 3), kwargs=dict(key='value'))

Tasks can be run by executing the management command flush_queue:

python manage.py flush_queue

async.schedule

schedule(function, args = None, kwargs = None, run_after= None, meta = None)

Returns a Job instance that is used to record the task in the database. The job is a callable, and calling it will execute the task. Don’t do this directly until you’ve fullly understood how transactions are handled

• function Either the fully qualified name of the function that is to be run, or the function itself.

• args A tuple or list of arguments to be given to the function.

• kwargs A dict containing key word arguments to be passed to the function.

• run_after The earliest time that the function should be run.

• meta Parameters for controlling how the function is to be executed.

Transaction handling

Database transactions are hard to get right, and unfortunately Django doesn’t make them much easier. Firstly, you really want to be using a proper transactional database system.

Django has two major flaws when it comes to transaction handling:

1. The Django transaction functionality fails to create composable transactions.

2. The Django documentation makes a very poor recommendation about where to put the django.middleware.transaction.TransactionMiddleware.

The first problem is not going to get fixed in Django, but the second can be handled by

The path to happy transactions starts with the use of the transaction middleware described at https://docs.djangoproject.com/en/dev/topics/db/transactions/. This allows you to pretty much ignore transactions in your request handling code in Django. Note though that the Django documentation suggests that you put it in the wrong place. For best results you should always put it as early as you can, but certainly before things like the django.contrib.sessions.middleware.SessionMiddleware which will write to the database.

Within the async task execution each task is executed decorated by django.db.transaction.commit_on_success. This means that you cannot execute a task directly from within a page request if you are using the transaction middleware.

Doing development

This project uses git flow. Don’t forget to do ``git flow init -d``

To create virtual environments for running the tests you can execute test-projects/make-virtual-environments. To run the tests execute runtests.