Skip to main content

A transaction-aware Celery job setup

Project description

A transaction-aware Celery job setup. This is integrated with the Zope transaction package, which implements a full two-phase commit protocol. While it is not designed for anything other than Pyramid, it also does not use any component of Pyramid. It’s simply not tested anywhere else.


  • Queues tasks into a thread-local when they are called either using delay or apply_async.
  • If the transaction is aborted, then the tasks will never be called.
  • If the transaction is committed, the tasks will go through their normal apply_async process and be queued for processing.


Currently, the code is designed around Celery v3.1, and it is unknown whether it will work with previous versions. I’m more than happy to integrate changes that would make it work with other releases, but since I generally stay on the latest release, it isn’t a priority for my own development.


Using the library is a relatively easy thing to do. First, you’ll need to integrate Celery into your Pyramid application, for which I recommend using pyramid_celery. Once that’s done, you simply need to start creating your tasks. The big difference is for function-based tasks, you use a different decorator:

from pyramid_transactional_celery import task_tm

def add(x, y):
    """Add two numbers together."""
    return x + y

That’s all there is to it. For class-based tasks, you simply need to subclass TransactionalTask instead of Task:

from pyramid_transactional_celery import TransactionalTask

class SampleTask(TransactionalTask):
    """A sample task that is transactional."""
    def run(x, y):
        return x + y

That’s it. Bob’s your uncle.


0.1.1 (2015-01-19)

  • Removed an excess creation of a CeleryDataManager that was a left-over from a previous approach. While this didn’t create a bug, it wasted memory.

0.1.0 (2015-01-19)

  • Initial functionality, but more testing of edge cases is needed to ensure that it works correctly in all cases, and with other versions of Celery.
  • First release on PyPI.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pyramid_transactional_celery, version 0.1.1
Filename, size File type Python version Upload date Hashes
Filename, size pyramid_transactional_celery-0.1.1.tar.gz (13.3 kB) File type Source Python version None Upload date Hashes View
Filename, size pyramid_transactional_celery-0.1.1-py2.py3-none-any.whl (6.3 kB) File type Wheel Python version 3.4 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page