Skip to main content

A Django Tasks backend which uses Celery as its underlying queue

Project description

Django Tasks Fennel

A Django Tasks backend which uses Celery as its underlying queue.

Django version support. This backend works on Django 5.2 and 6.0+, and adapts to whichever Tasks framework your Django version provides:

  • Django 6.0+ — uses the built-in django.tasks framework. No extra dependency is needed; define tasks with from django.tasks import task.
  • Django 5.2 — uses the standalone django-tasks package. Install it via the django-tasks extra (see below) and define tasks with from django_tasks import task.

Installation

On Django 6.0 and later:

python -m pip install django-tasks-fennel

On Django 5.2, also pull in the standalone django-tasks package via the extra:

python -m pip install "django-tasks-fennel[django-tasks]"

First, add django_tasks_fennel to your INSTALLED_APPS:

INSTALLED_APPS = [
    # ...
    "django_tasks_fennel",
]

Then, configure it as your TASKS backend:

TASKS = {
    "default": {
        "BACKEND": "django_tasks_fennel.CeleryBackend",
        "QUEUES": ["default"]
    }
}

You also need to configure Celery in your Django project (broker, result backend, etc.):

CELERY_BROKER_URL = "redis://localhost:6379/0"
CELERY_RESULT_BACKEND = "redis://localhost:6379/0"

You can review the Celery documentation for more information on how to configure Celery.

Note that all Celery configuration options must be specified in uppercase instead of lowercase, and start with CELERY_, so for example the broker_url setting becomes CELERY_BROKER_URL.

Usage

The Celery-based backend acts as an interface between Django's Tasks framework and Celery, allowing tasks to be defined and enqueued using the Tasks API, but sent to a Celery broker and executed by Celery workers.

Quickstart

Define a task with the task decorator. On Django 6.0+ import it from the built-in django.tasks; on Django 5.2 import it from the standalone django_tasks package instead (see the Django version support note above):

# my_app/tasks.py
from django.tasks import task  # Django 5.2: from django_tasks import task

@task()
def send_welcome_email(user_id: int) -> None:
    ...

Enqueue it from a view or anywhere in your Django code:

result = send_welcome_email.enqueue(user_id=42)

result is a TaskResult. When the worker has picked the task up and finished it, refresh and inspect:

result.refresh()

if result.is_finished:
    print(result.status)         # TaskResultStatus.SUCCESSFUL or .FAILED
    print(result.started_at, result.finished_at)
    if result.errors:
        print(result.errors[0].exception_class, result.errors[0].traceback)

Pass per-call overrides through using():

from datetime import timedelta
from django.utils import timezone

send_welcome_email.using(
    queue_name="emails",
    priority=50,
    run_after=timezone.now() + timedelta(minutes=5),
).enqueue(user_id=42)

Backend Capabilities

Feature Supported How
supports_defer (run_after) yes Celery eta
supports_async_task (coroutines) yes wrapped via async_to_sync
supports_priority (-100..100) yes mapped to Celery's 0..9; requires AMQP broker (RabbitMQ) for reliable ordering
supports_get_result / refresh() yes requires a Celery result backend

This backend bridges Django's Tasks framework to Celery; it doesn't expose Celery-specific primitives. If you need chains, groups, chords, or periodic tasks (beat), keep using plain @shared_task for those — both can coexist in the same project. Django Task names are namespaced under django_tasks: in Celery's registry (see Task Names in Celery), so there's no collision.

Celery App

A Celery app is included at django_tasks_fennel.app. It reads configuration from your Django settings with the CELERY_ prefix and auto-discovers tasks. You can use it directly, or configure your own Celery app as you normally would.

Running Workers

Start a Celery worker as usual:

DJANGO_SETTINGS_MODULE=<your_project.settings> celery -A django_tasks_fennel.app worker -l INFO

Task Names in Celery

Django Tasks are registered in Celery's task registry under a namespaced name to avoid collisions with unrelated @shared_task registrations that may share the same dotted path. The wire format is:

django_tasks:<module_path>

For example, a @task()-decorated my_app.tasks.send_email is registered (and routed) as django_tasks:my_app.tasks.send_email. You'll see this name in celery inspect registered, worker logs, and any external monitoring (Flower, etc.). This is the name to use in Celery routing rules (task_routes) if you need per-task overrides.

Result Backend

A Celery result backend is required for get_result() and refresh() to work; without one, a warning is raised during Django's system checks. Setting CELERY_RESULT_EXTENDED=True is recommended so worker_ids and attempts are populated on completed results (args and kwargs don't need it — they come from the side-channel on key-value backends).

TaskResult field availability

Which TaskResult fields are populated depends on how Celery is configured. Below, KV-backend means a key-value-style Celery result backend (Redis, memcached, cache+...://, filesystem, MongoDB); DB/RPC means db+...:// or rpc://.

Field Required configuration Notes
status (RUNNING during execution) result backend The wrapper explicitly writes STARTED from the worker, so works without CELERY_TASK_TRACK_STARTED.
finished_at, last_attempted_at (completed) result backend From Celery's date_done.
errors[*].traceback result backend The worker's serialized traceback string (AsyncResult.traceback).
errors[*].exception_class result backend The original exception class round-trips through Celery's serializer.
worker_ids, attempts result backend + CELERY_RESULT_EXTENDED=True Populated from AsyncResult.worker and retries.
started_at KV-backend Persisted via a side-channel key so it survives Celery overwriting the meta with the return value on completion. None on DB/RPC backends.
enqueued_at, pre-worker task / args / kwargs KV-backend Written by the same side-channel on enqueue(). Also lets get_result() reconstruct the Task even with CELERY_RESULT_EXTENDED=False. On DB/RPC backends enqueued_at is None and get_result() only works after the worker has stored the result (with CELERY_RESULT_EXTENDED=True).

Project details


Release history Release notifications | RSS feed

This version

0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

django_tasks_fennel-0.1.tar.gz (24.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

django_tasks_fennel-0.1-py3-none-any.whl (14.7 kB view details)

Uploaded Python 3

File details

Details for the file django_tasks_fennel-0.1.tar.gz.

File metadata

  • Download URL: django_tasks_fennel-0.1.tar.gz
  • Upload date:
  • Size: 24.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for django_tasks_fennel-0.1.tar.gz
Algorithm Hash digest
SHA256 be77d465318b49d8f3f2b48bc5d20e06a56fc46b6eb8eb5d02e282e260760970
MD5 664863b7b123e3cf9f5dcc733b1ebf51
BLAKE2b-256 423cc8af1b95ee0b4f59453f96136e70fcefe008f7da1a0dbdb77111cad614c4

See more details on using hashes here.

File details

Details for the file django_tasks_fennel-0.1-py3-none-any.whl.

File metadata

File hashes

Hashes for django_tasks_fennel-0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 d66cd094fec8e32dafaa7f24b730832dac58e4cb7a66502eb0b0f9edc61809c5
MD5 8fa083828a3dee48f5fbc37c07875a9d
BLAKE2b-256 b9925cb3fbbbb92eef7ea7417caf6d606bbd93fb502111926a7eed4242d7bfc0

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page