Skip to main content

Postgres Broker for Dramatiq Task Queue

Project description

=============================================
dramatiq-pg -- Postgres Broker for Dramatiq
=============================================

dramatiq_ is a simple task queue implementation for Python3. dramatiq-pg
provides a Postgres-based implementation of a dramatiq broker.

**The project is not feature complete yet.**


Features
========

- Super simple deployment.
- Stores messages in a single table.
- All data are wrapped in a dedicated schema.
- Uses LISTEN/NOTIFY to keep worker sync. No polling.
- Reliable thanks to Postgres MVCC.
- Using plain psycopg2. No ORM.


Installation
============

- Install dramatiq-pg package from PyPI::

pip install dramatiq-pg

- Apply dramatiq_pg/schema.sql file in your database::

psql -f dramatiq_pg/schema.sql

- Before importing actors, define global broker with a connection pool::

import dramatiq
import dramatiq_pg
import psycopg2.pool

pool = psycopg2.pool.ThreadedConnectionPool(0, 4, conninfo)
dramatiq.set_broker(dramatiq_pg.PostgresBroker(pool=pool))

Now declare/import actors and manage worker just like any `dramatiq setup
<https://dramatiq.io/guide.html>`_.


Roadmap
=======

- Process missed notifyes while resuming worker.
- Functionnal tests.
- Result storage as JSONb.
- Delayed task.


.. _dramatiq:: https://dramatiq.io/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dramatiq-pg-0.1.0.tar.gz (3.5 kB view hashes)

Uploaded Source

Built Distribution

dramatiq_pg-0.1.0-py3-none-any.whl (6.8 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page