A transaction-aware Celery job setup
A transaction-aware Celery job setup. This is integrated with the Zope transaction package, which implements a full two-phase commit protocol. While it is not designed for anything other than Pyramid, it also does not use any component of Pyramid. It’s simply not tested anywhere else.
- Free software: BSD license
- Documentation: https://pyramid_transactional_celery.readthedocs.org.
- Queues tasks into a thread-local when they are called either using delay or apply_async.
- If the transaction is aborted, then the tasks will never be called.
- If the transaction is committed, the tasks will go through their normal apply_async process and be queued for processing.
Currently, the code is designed around Celery v3.1, and it is unknown whether it will work with previous versions. I’m more than happy to integrate changes that would make it work with other releases, but since I generally stay on the latest release, it isn’t a priority for my own development.
Using the library is a relatively easy thing to do. First, you’ll need to integrate Celery into your Pyramid application, for which I recommend using pyramid_celery. Once that’s done, you simply need to start creating your tasks. The big difference is for function-based tasks, you use a different decorator:
from pyramid_transactional_celery import task_tm @task_tm def add(x, y): """Add two numbers together.""" return x + y
That’s all there is to it. For class-based tasks, you simply need to subclass TransactionalTask instead of Task:
from pyramid_transactional_celery import TransactionalTask class SampleTask(TransactionalTask): """A sample task that is transactional.""" def run(x, y): return x + y
That’s it. Bob’s your uncle.
- Removed an excess creation of a CeleryDataManager that was a left-over from a previous approach. While this didn’t create a bug, it wasted memory.
- Initial functionality, but more testing of edge cases is needed to ensure that it works correctly in all cases, and with other versions of Celery.
- First release on PyPI.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|File Name & Checksum SHA256 Checksum Help||Version||File Type||Upload Date|
|pyramid_transactional_celery-0.1.1-py2.py3-none-any.whl (6.3 kB) Copy SHA256 Checksum SHA256||3.4||Wheel||Jan 20, 2015|
|pyramid_transactional_celery-0.1.1.tar.gz (13.3 kB) Copy SHA256 Checksum SHA256||–||Source||Jan 20, 2015|