This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

A transaction-aware Celery job setup

Project Description

A transaction-aware Celery job setup. This is integrated with the Zope transaction package, which implements a full two-phase commit protocol. While it is not designed for anything other than Pyramid, it also does not use any component of Pyramid. It’s simply not tested anywhere else.

Features

  • Queues tasks into a thread-local when they are called either using delay or apply_async.
  • If the transaction is aborted, then the tasks will never be called.
  • If the transaction is committed, the tasks will go through their normal apply_async process and be queued for processing.

Limitations

Currently, the code is designed around Celery v3.1, and it is unknown whether it will work with previous versions. I’m more than happy to integrate changes that would make it work with other releases, but since I generally stay on the latest release, it isn’t a priority for my own development.

Usage

Using the library is a relatively easy thing to do. First, you’ll need to integrate Celery into your Pyramid application, for which I recommend using pyramid_celery. Once that’s done, you simply need to start creating your tasks. The big difference is for function-based tasks, you use a different decorator:

from pyramid_transactional_celery import task_tm

@task_tm
def add(x, y):
    """Add two numbers together."""
    return x + y

That’s all there is to it. For class-based tasks, you simply need to subclass TransactionalTask instead of Task:

from pyramid_transactional_celery import TransactionalTask

class SampleTask(TransactionalTask):
    """A sample task that is transactional."""
    def run(x, y):
        return x + y

That’s it. Bob’s your uncle.

History

0.1.1 (2015-01-19)

  • Removed an excess creation of a CeleryDataManager that was a left-over from a previous approach. While this didn’t create a bug, it wasted memory.

0.1.0 (2015-01-19)

  • Initial functionality, but more testing of edge cases is needed to ensure that it works correctly in all cases, and with other versions of Celery.
  • First release on PyPI.
Release History

Release History

This version
History Node

0.1.1

History Node

0.1.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
pyramid_transactional_celery-0.1.1-py2.py3-none-any.whl (6.3 kB) Copy SHA256 Checksum SHA256 3.4 Wheel Jan 20, 2015
pyramid_transactional_celery-0.1.1.tar.gz (13.3 kB) Copy SHA256 Checksum SHA256 Source Jan 20, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting