Skip to main content

Postgres-based distributed task processing library

Project description

Deployed to PyPI Documentation Status Continuous Integration Status Coverage Status MIT License Contributor Covenant

Procrastinate is an open-source Python 3.6+ distributed task processing library, leveraging PostgreSQL to store task definitions, manage locks and dispatch tasks. It can be used within both sync and async code.

In other words, from your main code, you call specific functions (tasks) in a special way and instead of being run on the spot, they’re scheduled to be run elsewhere, now or in the future.

Here’s an example

# mycode.py

# Make an app in your code
app = procrastinate.App(connector=procrastinate.AiopgConnector())

# Then define tasks
@app.task(queue="sums")
def sum(a, b):
    with open("myfile", "w") as f:
        f.write(str(a + b))

# Launch a job
sum.defer(a=3, b=5)

# Somewhere in your program, run a worker
worker = procrastinate.Worker(
    app=app,
    queues=["sums"]
)
worker.run()

The worker will run the job, which will create a text file named myfile with the result of the sum 3 + 5 (that’s 8).

Similarly, from the command line:

export PROCRASTINATE_APP="mycode.app"

# Launch a job
procrastinate defer mycode.sum '{"a": 3, "b": 5}'

# Run a worker
procrastinate worker sums

Lastly, you can use Procrastinate asynchronously too:

# Define asynchronous tasks using coroutine functions
@app.task(queue="sums")
async def sum(a, b):
    await asyncio.sleep(a + b)

# Launch a job asynchronously
await sum.defer_async(a=3, b=5)

# Somewhere in your program, run a worker asynchronously
worker = procrastinate.Worker(
    app=app,
    queues=["sums"]
)
await worker.run_async()

There are quite a few interesting features that Procrastinate adds to the mix. You can head to the Quickstart section for a general tour or to the How-To sections for specific features. The Discussion section should hopefully answer your questions. Otherwise, feel free to open an issue.

The project is still quite early-stage and will probably evolve.

Note to my future self: add a quick note here on why this project is namedProcrastinate”.

Where to go from here

The complete docs is probably the best place to learn about the project.

If you encounter a bug, or want to get in touch, you’re always welcome to open a ticket.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

procrastinate-0.12.0.tar.gz (44.8 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page