WebCase celery one task caller

Package to call celery tasks only once per unique key.

Installation

pip install wc-django-celery-unicall

In settings.py:

INSTALLED_APPS += [
  'wcd_celery_unicall',
]

Usage

from celery import Celery
from wcd_celery_unicall.task import CACHE_KEY_PREFIX
from wcd_celery_unicall.shortcuts import make_unicall, task

app = Celery('app')

# First after defining and configuring celery app you should change
# the default `Task` class.
# It has a small improvement in `apply_async` method, that will revoke
# other pending tasks that have the same uniqueness key.
app.Task = make_unicall(app.Task)

# ...

from django.core.cache import cache

# All other tasks that you want to check for uniqueness have to be defined
# using a `task` shortcut decorator.
# So instead of writing `@app.task()` write `@task(app)`.
# But that's not all. To enforce uniqueness check mechanics you must
# provide at least 2 additional parameters: `unicall_key` and `unicall_cache`.
@task(
  # Your celery app:
  app,
  # Function that receives the same arguments that your task, and generates
  # unique key to track task execution.
  unicall_key=lambda user_id, **_: f'sync_user {user_id}',
  # And a django cache backend implementation. This could be anything you
  # wish, but `LockMem` cache backend, for example will be useless.
  # So this should be "global" cache backend, like redis or memcached.
  unicall_cache=cache,
  # Optional. Additional TTL, that will be added to key expirity.
  unicall_additional_ttl=None,
  # Optional. Cache key prefix, if you don't like the default one.
  unicall_key_prefix=CACHE_KEY_PREFIX,
)
def do_something(user_id: int, param2: bool = True, param_3=None):
  pass

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

[0.1.1]

Fixed

• Latest task identifier storage and check.

Changed

• Increased additional ttl to remember last task id for longer.

[0.1.0]

Initial version.