Skip to main content

Postgres-based distributed task processing library

Project description

Procrastinate: PostgreSQL-based Task Queue for Python

Deployed to PyPI Deployed to PyPI GitHub Repository Continuous Integration Documentation Coverage badge MIT License Contributor Covenant Discord

Procrastinate is looking for additional maintainers!

Procrastinate is an open-source Python 3.9+ distributed task processing library, leveraging PostgreSQL 13+ to store task definitions, manage locks and dispatch tasks. It can be used within both sync and async code, has Django integration, and is easy to use with ASGI frameworks. It supports periodic tasks, retries, arbitrary task locks etc.

In other words, from your main code, you call specific functions (tasks) in a special way and instead of being run on the spot, they're scheduled to be run elsewhere, now or in the future.

Here's an example (if you want to run the code yourself, head to Quickstart):

# mycode.py
import procrastinate

# Make an app in your code
app = procrastinate.App(connector=procrastinate.SyncPsycopgConnector())

# Then define tasks
@app.task(queue="sums")
def sum(a, b):
    with open("myfile", "w") as f:
        f.write(str(a + b))

with app.open():
    # Launch a job
    sum.defer(a=3, b=5)

# Somewhere in your program, run a worker (actually, it's usually a
# different program than the one deferring jobs for execution)
app.run_worker(queues=["sums"])

The worker will run the job, which will create a text file named myfile with the result of the sum 3 + 5 (that's 8).

Similarly, from the command line:

export PROCRASTINATE_APP="mycode.app"

# Launch a job
procrastinate defer mycode.sum '{"a": 3, "b": 5}'

# Run a worker
procrastinate worker -q sums

Lastly, you can use Procrastinate asynchronously too (actually, it's the recommended way to use it):

import asyncio

import procrastinate

# Make an app in your code
app = procrastinate.App(connector=procrastinate.PsycopgConnector())

# Define tasks using coroutine functions
@app.task(queue="sums")
async def sum(a, b):
    await asyncio.sleep(a + b)

async with app.open_async():
    # Launch a job
    await sum.defer_async(a=3, b=5)

    # Somewhere in your program, run a worker (actually, it's often a
    # different program than the one deferring jobs for execution)
    await app.run_worker_async(queues=["sums"])

There are quite a few interesting features that Procrastinate adds to the mix. You can head to the Quickstart section for a general tour or to the How-To sections for specific features. The Discussion section should hopefully answer your questions. Otherwise, feel free to open an issue.

Note to my future self: add a quick note here on why this project is named "Procrastinate" ;) .

Where to go from here

The complete docs is probably the best place to learn about the project.

If you encounter a bug, or want to get in touch, you're always welcome to open a ticket.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

procrastinate-3.4.0.tar.gz (82.8 kB view details)

Uploaded Source

Built Distribution

procrastinate-3.4.0-py3-none-any.whl (146.6 kB view details)

Uploaded Python 3

File details

Details for the file procrastinate-3.4.0.tar.gz.

File metadata

  • Download URL: procrastinate-3.4.0.tar.gz
  • Upload date:
  • Size: 82.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for procrastinate-3.4.0.tar.gz
Algorithm Hash digest
SHA256 9cd3af413f351c88da40681d3d88b6df8904c8a3423aec87687c0b7eb7d684a4
MD5 dbe3504d8cc8289b611322dfd8461cc5
BLAKE2b-256 4440d6b54478a99ec88b4c5ad2d5e1deec639c12a29940932f16962ea4d3023f

See more details on using hashes here.

Provenance

The following attestation bundles were made for procrastinate-3.4.0.tar.gz:

Publisher: ci.yml on procrastinate-org/procrastinate

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file procrastinate-3.4.0-py3-none-any.whl.

File metadata

  • Download URL: procrastinate-3.4.0-py3-none-any.whl
  • Upload date:
  • Size: 146.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for procrastinate-3.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 dbcca697dc78d8b71873692e3567de482db076bf06f55af775c09aa82a3e1066
MD5 3c7bf39a6f4663638ae9890b968d699f
BLAKE2b-256 e1def81bcfd3237ab7c6441d3ec7ad665cfaf900185d8cb066278ab0e7e7ee15

See more details on using hashes here.

Provenance

The following attestation bundles were made for procrastinate-3.4.0-py3-none-any.whl:

Publisher: ci.yml on procrastinate-org/procrastinate

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page