Transaction barriers for Django and Celery.
Project description
django_transaction_barrier provides a barrier-like abstraction for transactions. A Django application developer can use a “transaction barrier” to spawn a task within a transaction and guarantee that the task blocks until it’s able to access the updates made in the transaction. django_transaction_barrier is designed with Celery in mind and provides a Celery task base class. Using the base class it’s easy to write code that atomically modifies the database and spawns a Celery task that executes after the transaction commits.
Installation
Install from source or use pip:
pip install django_transaction_barrier
and add “django_transaction_barrier” to INSTALLED_APPS in settings.py:
INSTALLED_APPS = (
'django_transaction_barrier',
...
)
Usage
from celery import task
from django.db import transaction
from django_transaction_barrier.celery import TransactionBarrierTask
@task(base=TransactionBarrierTask)
def do_something_task(model_id):
value = Model.objects.get(id=model_id).value
...
@transaction.atomic
def kick_off_task(model, value):
model.value = value
do_something_task.apply_async_with_barrier(args=(model.id,))
model.save()
Details
If an application spawns an asynchronous TransactionBarrierTask the task is guaranteed to execute eventually (assuming a durable task queue) after the transaction commits. If the transaction aborts, the task raises a TransactionAborted exception and does not execute. In autocommit mode (i.e., “outside of a transaction”) TransactionBarrierTasks behave like normal Celery tasks.
If an application synchrnously executes a TransactionBarrierTask (e.g., with Celery eager mode) within a transaction, the task executes immediately without waiting for the transaction to commit.
Implementation
django_transaction_barrier implements transaction barriers using row insertion to signify a committed transaction and some DB-specifc logic to detect an abort.
Tests
docker build -t tests . && docker run tests
TODO
Add support for a mysql backend.