A lightweight distributed queue using a postgres DB.
Project description
lonny_common_pg_queue
A lightweight distributed queue using a postgres DB.
Installation
pip install lonny_common_pg_queue
Usage
The first step is importing the Queue
class. This can be done as:
from lonny_common_pg_queue import Queue
We then need to ensure the requisite queue table is implemented. This can be done by passing a lonny_common_pg
connection object to the Queue.setup(db)
method (This method is idempotent and can be run multiple times).
To create a queue
object simply invoke the constructor with a lonny_common_pg
connection and a mandatory namespace
:
queue_1 = Queue(db, "queue_1")
queue_2 = Queue(db, "queue_2")
The messages placed in these queues will be isolated from one another.
To enqueue objects, simply call queue.put(obj)
on any JSON serializable object.
To retrieve an object from the queue, call queue.get()
. This will return a Message
object. The underlying payload can be accessed via message.payload
. Messages work similar to how they work in AWS SQS - messages retrieved from the queue are immediately "locked". In this state they cannot be retrieved by another consume. After a timeout, they become unlocked and can be accessed as before by another consumer. This happens a maximum number of times until the message is deleted permanently.
Once you are done with a message, you should call .consume()
on it - this will prevent it being accessed again at a later point. To simplify this procedure, we can use the with
syntax:
with queue.get() as payload:
do_stuff(payload)
If we leave the block with no exception, the message is consumed/deleted as expected. However, if something goes wrong, consume
isn't called, giving us another chance in the future to handle this message.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for lonny_common_pg_queue-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b643d571263499f98fafc45666e0d1c5c4378c8a74c63a89c2f41cafa6755949 |
|
MD5 | 89e18d5baa332a5e935d54ad816e6511 |
|
BLAKE2b-256 | 2b732206edf241b0cec360169cf7106d0ad2ab2896ef22afec4da865cc6d2fe7 |
Hashes for lonny_common_pg_queue-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f3e16800398ed4f70fe905c1b8b2512758c80f2dfe4ec2e3c09e8a000419a6e |
|
MD5 | 3a6672ca79d9dca17e5e569aa8d8eac2 |
|
BLAKE2b-256 | b128bdfc75df3e93de5d1ce32fc723033cf229a9f61ea17290e9edbb55bbe33d |