Postgres Broker for Dramatiq Task Queue
dramatiq-pg − Postgres Broker for Dramatiq
dramatiq is a simple task queue implementation for Python3. dramatiq-pg provides a Postgres-based implementation of a dramatiq broker.
The project is not feature complete yet.
- Super simple deployment.
- Uses plain psycopg2. No ORM.
- Stores message payload as native JSONb.
- Stores all messages in a single table, in a dedicated schema.
- Uses LISTEN/NOTIFY to keep worker sync. No polling.
- Replay pending messages on worker startup.
- Requeues failed tasks.
- Reliable thanks to Postgres MVCC.
- Self-healing. Old messages are purge from time to time.
Note that dramatiq assumes tasks are idempotent. This broker makes the same assumptions for recovering after a crash.
- Install dramatiq-pg package from PyPI:
$ pip install dramatiq-pg
- Apply dramatiq_pg/schema.sql file in your database:
$ psql -f dramatiq_pg/schema.sql
- Before importing actors, define global broker with a connection
import dramatiq import dramatiq_pg import psycopg2.pool pool = psycopg2.pool.ThreadedConnectionPool(0, 4, conninfo) dramatiq.set_broker(dramatiq_pg.PostgresBroker(pool=pool))
The CLI tool
dramatiq-pg allows you to show stats on the queue and purge old
--help for details.
- Result storage as JSONb.
- Delayed task.
Feel free to suggest feature through support channels.
If you encounter a bug or miss a feature, please open an issue on GitLab with as much information as possible.
dramatiq_pg is available under the PostgreSQL licence.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size dramatiq_pg-0.3.0-py3-none-any.whl (15.9 kB)||File type Wheel||Python version py3||Upload date||Hashes View hashes|
|Filename, size dramatiq-pg-0.3.0.tar.gz (6.8 kB)||File type Source||Python version None||Upload date||Hashes View hashes|
Hashes for dramatiq_pg-0.3.0-py3-none-any.whl