Procrastinate: PostgreSQL-based Task Queue for Python

Procrastinate is looking for additional maintainers!

Procrastinate is an open-source Python 3.8+ distributed task processing library, leveraging PostgreSQL to store task definitions, manage locks and dispatch tasks. It can be used within both sync and async code and integrated to Django.

In other words, from your main code, you call specific functions (tasks) in a special way and instead of being run on the spot, they’re scheduled to be run elsewhere, now or in the future.

Here’s an example (if you want to run the code yourself, head to Quickstart):

# mycode.py
import procrastinate

# Make an app in your code
app = procrastinate.App(connector=procrastinate.AiopgConnector())

# Then define tasks
@app.task(queue="sums")
def sum(a, b):
    with open("myfile", "w") as f:
        f.write(str(a + b))

with app.open():
    # Launch a job
    sum.defer(a=3, b=5)

    # Somewhere in your program, run a worker (actually, it's often a
    # different program than the one deferring jobs for execution)
    app.run_worker(queues=["sums"])

The worker will run the job, which will create a text file named myfile with the result of the sum 3 + 5 (that’s 8).

Similarly, from the command line:

export PROCRASTINATE_APP="mycode.app"

# Launch a job
procrastinate defer mycode.sum '{"a": 3, "b": 5}'

# Run a worker
procrastinate worker -q sums

Lastly, you can use Procrastinate asynchronously too:

import asyncio

import procrastinate

# Make an app in your code
app = procrastinate.App(connector=procrastinate.AiopgConnector())

# Define tasks using coroutine functions
@app.task(queue="sums")
async def sum(a, b):
    await asyncio.sleep(a + b)

async with app.open_async():
    # Launch a job
    await sum.defer_async(a=3, b=5)

    # Somewhere in your program, run a worker (actually, it's often a
    # different program than the one deferring jobs for execution)
    await app.run_worker_async(queues=["sums"])

There are quite a few interesting features that Procrastinate adds to the mix. You can head to the Quickstart section for a general tour or to the How-To sections for specific features. The Discussion section should hopefully answer your questions. Otherwise, feel free to open an issue.

The project is still quite early-stage and will probably evolve.

Note to my future self: add a quick note here on why this project is named “Procrastinate”.

Where to go from here

The complete docs is probably the best place to learn about the project.

If you encounter a bug, or want to get in touch, you’re always welcome to open a ticket.

Metadata

Release files for procrastinate 0.29.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for procrastinate 0.29.1
File	Size	Uploaded
procrastinate-0.29.1.tar.gz	59.0 kB	Dec 13, 2023	Details

Built distribution (wheel)

Table of built distributions (wheels) for procrastinate 0.29.1
File	Interpreter	ABI	Platform	Reset
procrastinate-0.29.1-py3-none-any.whl 83.0 kB Dec 13, 2023	Python 3	none	any	Details

Total release size: 142.0 kB

Release files / procrastinate-0.29.1.tar.gz

Download URL	procrastinate-0.29.1.tar.gz
Size	59.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`79addcf531e3633579abe6dd8b8f9cbe98d28a2643f3f8e9c967fa5dcb9924e7`
BLAKE2b-256 checksum How to use checksums	`18ce9d13b692b5aae1410fcf2a8baaea445de9d5e8b51f43726d7de32661649b`
Upload date	Dec 13, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.7.1 CPython/3.10.12 Linux/6.2.0-1018-azure`

Release files / procrastinate-0.29.1-py3-none-any.whl

Download URL	procrastinate-0.29.1-py3-none-any.whl
Size	83.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`3565f8ab1ab2e0f954b9de5c469e32df5426b14ce59f7ab4460b49f0206dfce7`
BLAKE2b-256 checksum How to use checksums	`fe53b53d45ec71468b504cf0af1f012b8bbed50d5fe6dcb819cfc0d73add00a2`
Upload date	Dec 13, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.7.1 CPython/3.10.12 Linux/6.2.0-1018-azure`