simple-module-background-tasks 0.0.21

Celery + Redis task queue with admin UI for monitoring and retrying failed/stuck tasks

simple_module_background_tasks

Celery + Redis background-task module for simple_module apps. Provides a pre-configured Celery instance, a task registration hook, and an admin UI for monitoring + retrying failed/stuck tasks.

Install

pip install simple_module_background_tasks

Requires a Redis broker. The broker / result-backend URLs are module settings (broker_url, default redis://localhost:6379/0; result_backend, default redis://localhost:6379/1) configured from the DB settings store via the admin UI — they are not read from environment variables at runtime. These fields are requires_restart (changing them needs a worker/web restart).

What it provides

• Zero-config task discovery — any installed module that ships a tasks.py has its tasks autodiscovered (celery.autodiscover_tasks imports <package>.tasks for every installed module). No per-module registration hook.
• Admin UI at /admin/background-tasks — list recent runs, retry failed, inspect tracebacks (gated by the background_tasks.view permission).
• build_celery(settings) factory in background_tasks.celery_app, plus bind_task_context / get_log_context / install_log_filter exported from the package root (import name background_tasks, distribution name simple_module_background_tasks).

Usage

Declare a task in a module's tasks.py with Celery's @shared_task — it's autodiscovered, no registration hook needed:

# modules/reports/reports/tasks.py
from celery import shared_task


@shared_task(name="reports.generate")
def generate_report(report_id: int) -> None:
    ...

Declaring background_tasks as a depends_on ensures the Celery app is built before your tasks run:

class ReportsModule(ModuleBase):
    meta = ModuleMeta(name="reports", depends_on=["background_tasks"])

Enqueue from an endpoint:

generate_report.delay(report_id=42)

Run a worker locally (the scaffolded host ships a scripts/run_worker.py that builds the app via build_celery):

uv run celery -A scripts.run_worker:celery worker -l info
uv run celery -A scripts.run_worker:celery beat   -l info

Worker log context

Every worker log line automatically carries the Celery task identifiers that fired it. A LogContextFilter is attached when the Celery app is built (build_celery) and the task_prerun / task_postrun signals bind task_id + task_name into contextvars for the task's duration:

{"level": "INFO", "logger": "reports.tasks", "message": "ingest done",
 "task_id": "9c2a…", "task_name": "reports.generate"}

Use bind_task_context(...) to attach app-level identifiers (the domain job_id that named a Celery task is the canonical example):

from background_tasks import bind_task_context
from celery import shared_task

@shared_task
def process_dataset(job_id: int) -> None:
    with bind_task_context(job_id=job_id):
        logger.info("starting ingest")   # now carries job_id too

Bindings nest cleanly and restore on exit. structlog users can mount the same contextvars directly via structlog.contextvars.merge_contextvars.

Depends on

• simple_module_core, simple_module_db, simple_module_hosting, simple_module_settings
• celery[redis]>=5.4, redis>=5

License

MIT — see LICENSE.