django-ox 0.2.0

A database-backed worker backend for Django's Tasks framework (django.tasks, Django 6.0+).

Documentation: https://oxpull.com/django-ox/

Django 6.0 ships the Tasks API but no production backend: the built-in ImmediateBackend and DummyBackend are for development and testing only. django-ox stores background tasks in the database you already run and executes them with a worker process. There is no broker to provision, secure, upgrade or back up.

One fewer service to run

A broker-based task queue adds a second datastore to your deployment. Redis or RabbitMQ has to be provisioned, monitored, secured and upgraded, and it has to be running before a single task executes. For an application that already depends on a database, that is a full operational surface added for one feature.

django-ox uses the database you already run. A deployment is your application, a worker process, and one migration. Backups already cover the queue, because the queue is a table.

Transactional enqueue

enqueue() is a single INSERT on your default database connection, so it participates in the caller's open transaction. A task enqueued inside transaction.atomic() becomes visible to workers only when the transaction commits, and disappears on rollback. There is no window where business data exists without its task, or a task without its data, and no transaction.on_commit() boilerplate. Execution is at-least-once: workers claim tasks with SELECT ... FOR UPDATE SKIP LOCKED on databases that support it (PostgreSQL, MySQL 8+) and an atomic compare-and-set UPDATE elsewhere (including SQLite), and a reaper returns tasks whose worker died to the queue. Failed tasks retry with exponential backoff up to a configurable attempt limit, keeping the full traceback of every attempt.

Install

Requires Python 3.12+ and Django 6.0+.

pip install django-ox

INSTALLED_APPS = [
    # ...
    "django_ox",
]

TASKS = {
    "default": {
        "BACKEND": "django_ox.backend.OxBackend",
        "QUEUES": ["default", "emails"],  # [] allows any queue name
        "OPTIONS": {
            "MAX_ATTEMPTS": 3,  # executions per task before FAILED
            "LOCK_TIMEOUT": 300,  # seconds before a dead worker's task is reclaimed
            "BACKOFF_INITIAL": 5,  # first retry delay, seconds; doubles per attempt
            "BACKOFF_MAX": 600,  # retry delay ceiling, seconds
        },
    }
}

Then run migrations:

python manage.py migrate django_ox

Quickstart

Tasks are plain django.tasks tasks; django-ox adds nothing to learn on the producer side.

from django.tasks import task


@task
def send_welcome_email(user_id): ...


result = send_welcome_email.enqueue(user_id=42)
result.refresh()  # later: status, return_value, errors

Run a worker:

python manage.py ox_worker

Worker CLI

On SIGTERM or SIGINT the worker stops claiming, finishes in-flight tasks, then exits. A second signal forces an immediate exit.

Pruning

Finished task rows stay in the table until pruned. Run ox_prune on your own schedule (cron, systemd timer):

python manage.py ox_prune --older-than 7d

Only SUCCESSFUL rows (and, with --include-failed, FAILED rows) past the cutoff are deleted. READY and RUNNING rows are never touched, whatever their age. Old rows from the recurring-schedule tick log are cleared with the same cutoff, always keeping each schedule's most recent tick.

Health and monitoring

django_ox.stats exposes queue metrics as plain functions, each a single ORM query: per-queue status counts, backlog depth and age, throughput, and failure rate. The ox_health command turns thresholds on those numbers into an exit code for cron alerting and container probes:

python manage.py ox_health --max-backlog 1000 --max-age 600

Worker lifecycle events (claim, start, success, retry, failure, reclaim, shutdown) log to the django_ox logger with stable extra keys (task id, queue, attempt, duration), ready for JSON log handlers.

Recurring tasks

Schedules are declared in settings, next to the backend they enqueue through, and deploy with your code. There are no rows to edit by hand and no separate scheduler process to keep alive:

TASKS = {
    "default": {
        "BACKEND": "django_ox.backend.OxBackend",
        "QUEUES": ["default", "emails"],
        "OPTIONS": {
            "SCHEDULES": {
                "nightly-report": {
                    "task": "reports.tasks.build_report",
                    "cron": "0 3 * * *",
                    "kwargs": {"full": True},
                },
                "warm-cache": {
                    "task": "core.tasks.warm_cache",
                    "cron": "*/15 * * * *",
                },
            },
        },
    }
}

Each tick enqueues a normal task instance, which workers claim and execute through the ordinary queue: retries, backoff, priorities and the result store all apply unchanged. Every running worker doubles as the scheduler, and a unique constraint on (schedule name, tick time) makes each tick fire exactly once however many workers are polling.

Cron expressions use the classic five-field syntax: *, lists (1,15), ranges (mon-fri), steps (*/15), month and weekday names, 0 or 7 for Sunday, and the @hourly, @daily, @weekly, @monthly and @yearly shortcuts. When both day-of-month and day-of-week are restricted, a day matches if either field does, as in vixie cron. Times are wall-clock in your TIME_ZONE.

Misconfigured schedules (a task path that does not import, an expression that can never fire) fail at worker startup and in manage.py check, not silently at dispatch time.

Missed ticks: if every worker was down when a tick passed, the latest missed tick fires once on recovery and older ones are skipped, so a nightly job still runs after an unlucky deploy window but a backlog never stampedes. A newly deployed schedule waits for its next tick rather than firing for a time before it existed.

Behavior details

• run_after (deferred tasks), priority (-100 to 100, higher runs first), get_result() and the async variants are all supported; the backend declares supports_defer, supports_priority, supports_get_result and supports_async_task accordingly.
• Retry state is visible in the database: attempts, per-attempt tracebacks, and the next scheduled run (run_after).
• Because execution is at-least-once, tasks should be idempotent. A task is retried both when it raises and when its worker dies mid-run.
• Concurrency uses a thread pool. That fits I/O-bound tasks (email, HTTP, ORM); for CPU-bound work, run multiple worker processes with --concurrency=1 instead.

Scope and roadmap

The core is deliberately small: a durable queue, a worker, recurring schedules, monitoring, and nothing else to operate. Outside the current scope: task revocation after enqueue, and multi-database routing (tasks are stored on the default database for the model).

Batches and unique tasks ship in Oxpull Pro; the waitlist is at https://oxpull.com/. Metrics stay in this package: django_ox.stats and ox_health are free and stay free. Rate limiting is in neither tier.

Stability

What counts as public API, the pre-1.0 versioning and deprecation policy, and the supported Python and Django versions are documented in the stability policy.

License

BSD 3-Clause.