dramatiq-pg

Dramatiq-pg

Dramatiq is a simple task queue implementation for Python3. dramatiq-pg provides a Postgres-based implementation of a dramatiq broker.

Features

Super simple deployment: Single table, no ORM.
Stores message payload and results as native JSONb.
Uses LISTEN/NOTIFY to keep worker sync. No polling.
Implements delayed task.
Reliable thanks to Postgres MVCC.
Self-healing: automatic purge of old messages. Automatic recovery after crash.
Utility CLI for maintainance: flush, purge, stats, etc.

Note that dramatiq assumes tasks are idempotent. This broker makes the same assumptions for recovering after a crash.

Installation

Install dramatiq-pg package from PyPI:
```
$ pip install dramatiq-pg psycopg2-binary
```
Ensure you have either psycopg2 or psycopg2-binary installed.
Init database schema with init command.
```
$ dramatiq-pg init
```
Or adapt dramatiq-pg/schema.sql to your needs.

Before importing actors, define global broker with a connection pool:

import dramatiq
import psycopg2.pool
from dramatiq_pg import PostgresBroker

dramatiq.set_broker(PostgresBroker(i))

@dramatiq.actor
def myactor():
    ...

Now declare/import actors and manage worker just like any dramatiq setup. An example script is available, tested on CI.

The CLI tool dramatiq-pg allows you to requeue messages, purge old messages and show stats on the queue. See --help for details.

Dramatiq-pg documentation is hosted on GitLab and give you more details on deployment and operation of Postgres as a Dramatiq broker.

Integration

Django : Use django-dramatiq-pg by Curtis Maloney. It includes configuration, ORM model and database migration.

Support

If you encounter a bug or miss a feature, please open an issue on GitLab with as much information as possible.

dramatiq_pg is available under the PostgreSQL licence.

Credit

Thanks to all contributors :

Andy Freeland
Curtis Maloney, Django support.
Federico Caselli, bugfixes.
Giuseppe Papallo, bugfixes.
Rafal Kwasny, improvements.

The logo is a creation of Damien CAZEILS

Metadata

Release files for dramatiq-pg 0.12.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dramatiq-pg 0.12.0
File	Size	Uploaded	Provenance
dramatiq_pg-0.12.0.tar.gz	13.7 kB	Jul 24, 2024	Without provenance	Details

Built distribution (wheel)

Table of built distributions (wheels) for dramatiq-pg 0.12.0
File	Interpreter	ABI	Platform	Reset
dramatiq_pg-0.12.0-py3-none-any.whl 15.0 kB Jul 24, 2024 Without provenance	Python 3	none	any	Details

Total release size: 28.7 kB

Release files / dramatiq_pg-0.12.0.tar.gz

Download URL	dramatiq_pg-0.12.0.tar.gz
Size	13.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`828245fcf3700f6098ec1bff9eb13e50acff69d9c490ceeec5b69ee75d0f9fb1`
BLAKE2b-256 checksum How to use checksums	`949885c41e5dce7aece26641b28d82fd306325c49f7a92f1ccce0866308462c5`
Upload date	Jul 24, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.8.3 CPython/3.11.1 Linux/6.1.0-22-amd64`

Release files / dramatiq_pg-0.12.0-py3-none-any.whl

Download URL	dramatiq_pg-0.12.0-py3-none-any.whl
Size	15.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`808bbe854d1266922570bd5afeacac0ddb93f6a7dbf68624dde2f76d72d95b58`
BLAKE2b-256 checksum How to use checksums	`02cc59114fca251d37c980b048bdb837fbdbf02ec8d2d64dd653c78628e6afd1`
Upload date	Jul 24, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.8.3 CPython/3.11.1 Linux/6.1.0-22-amd64`

Security for dramatiq-pg 0.12.0

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.