Skip to main content

Postgres Broker for Dramatiq Task Queue

Project description

=============================================
dramatiq-pg -- Postgres Broker for Dramatiq
=============================================

dramatiq_ is a simple task queue implementation for Python3. dramatiq-pg
provides a Postgres-based implementation of a dramatiq broker.

**The project is not feature complete yet.**


Features
========

- Super simple deployment.
- Stores messages in a single table.
- All data are wrapped in a dedicated schema.
- Uses LISTEN/NOTIFY to keep worker sync. No polling.
- Reliable thanks to Postgres MVCC.
- Using plain psycopg2. No ORM.


Installation
============

- Install dramatiq-pg package from PyPI::

pip install dramatiq-pg

- Apply dramatiq_pg/schema.sql file in your database::

psql -f dramatiq_pg/schema.sql

- Before importing actors, define global broker with a connection pool::

import dramatiq
import dramatiq_pg
import psycopg2.pool

pool = psycopg2.pool.ThreadedConnectionPool(0, 4, conninfo)
dramatiq.set_broker(dramatiq_pg.PostgresBroker(pool=pool))

Now declare/import actors and manage worker just like any `dramatiq setup
<https://dramatiq.io/guide.html>`_.


Roadmap
=======

- Process missed notifyes while resuming worker.
- Functionnal tests.
- Result storage as JSONb.
- Delayed task.


.. _dramatiq:: https://dramatiq.io/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for dramatiq-pg, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size dramatiq_pg-0.1.0-py3-none-any.whl (6.8 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size dramatiq-pg-0.1.0.tar.gz (3.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page