Postgres Broker for Dramatiq Task Queue
Dramatiq is a simple task queue implementation for Python3. dramatiq-pg provides a Postgres-based implementation of a dramatiq broker.
- Super simple deployment: Single table, no ORM.
- Stores message payload and results as native JSONb.
- Uses LISTEN/NOTIFY to keep worker sync. No polling.
- Implements delayed task.
- Reliable thanks to Postgres MVCC.
- Self-healing: automatic purge of old messages. Automatic recovery after crash.
- Utility CLI for maintainance: flush, purge, stats, etc.
Note that dramatiq assumes tasks are idempotent. This broker makes the same assumptions for recovering after a crash.
- Install dramatiq-pg package from PyPI:
$ pip install dramatiq-pg psycopg2-binaryEnsure you have either psycopg2 or psycopg2-binary installed.
- Init database schema with
$ dramatiq-pg initOr adapt
dramatiq-pg/schema.sqlto your needs.
- Before importing actors, define global broker with a connection
import dramatiq import psycopg2.pool from dramatiq_pg import PostgresBroker dramatiq.set_broker(PostgresBroker(i)) @dramatiq.actor def myactor(): ...
Now declare/import actors and manage worker just like any dramatiq setup. An example script is available, tested on CI.
The CLI tool
dramatiq-pg allows you to requeue messages, purge old messages
and show stats on the queue. See
--help for details.
Dramatiq-pg documentation is hosted on GitLab and give you more details on deployment and operation of Postgres as a Dramatiq broker.
Django : Use django-dramatiq-pg by Curtis Maloney. It includes configuration, ORM model and database migration.
If you encounter a bug or miss a feature, please open an issue on GitLab with as much information as possible.
dramatiq_pg is available under the PostgreSQL licence.
Thanks to all contributors :
- Andy Freeland
- Curtis Maloney, Django support.
- Federico Caselli, bugfixes.
- Giuseppe Papallo, bugfixes.
- Rafal Kwasny, improvements.
The logo is a creation of Damien CAZEILS
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for dramatiq_pg-0.11.0-py3-none-any.whl