PgQueuer is a Python library leveraging PostgreSQL for efficient job queuing.
Project description
🚀 PgQueuer - Building Smoother Workflows One Queue at a Time 🚀
📚 Documentation: Explore the Docs 📖
🔍 Source Code: View on GitHub 💾
PgQueuer
PgQueuer is a minimalist, high-performance job queue library for Python, leveraging the robustness of PostgreSQL. Designed for simplicity and efficiency, PgQueuer uses PostgreSQL's LISTEN/NOTIFY to manage job queues effortlessly.
Features
- Simple Integration: Easy to integrate with existing Python applications using PostgreSQL.
- Efficient Concurrency Handling: Utilizes PostgreSQL's
FOR UPDATE SKIP LOCKED
for reliable and concurrent job processing. - Real-time Notifications: Leverages
LISTEN
andNOTIFY
for real-time updates on job status changes.
Installation
To install PgQueuer, simply install with pip the following command:
pip install PgQueuer
Example Usage
Here's how you can use PgQueuer in a typical scenario processing incoming data messages:
import asyncio
import asyncpg
from PgQueuer.models import Job
from PgQueuer.qm import QueueManager
async def main() -> None:
pool = await asyncpg.create_pool(min_size=2)
qm = QueueManager(pool)
N = 1_000
# Enqueue messages.
for n in range(N):
await qm.queries.enqueue("fetch", f"this is from me: {n}".encode())
@qm.entrypoint("fetch")
async def process_message(job: Job) -> None:
print(f"Processed message: {job}")
await qm.run()
if __name__ == "__main__":
asyncio.run(main())
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pgqueuer-0.4.0b1.tar.gz
(28.1 kB
view hashes)
Built Distribution
PgQueuer-0.4.0b1-py3-none-any.whl
(17.0 kB
view hashes)
Close
Hashes for PgQueuer-0.4.0b1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a09e45a2af5f65fc75acc83ff4ef49a5cc7597fe35c7acd9e166e302d45c3083 |
|
MD5 | 8dd0375774dd08b888c8baa3c717acbf |
|
BLAKE2b-256 | 0355009240504c85aac72a18088d987b81fcce2a61a3372d7c64e640a9fd359a |