Celery Board

A dashboard for Celery, built on Flower and inspired by bull-board.

It keeps Flower's core as it is: event capture, worker inspection, task search and Prometheus metrics. On top of that it puts a new UI, which you can serve with Tornado, Starlette/FastAPI or aiohttp.

Failed tasks	Worker

Features

• Tasks: bull-board style cards for every state: pending, received, started, retry, succeeded, failed, revoked and rejected. Each card has a timeline, arguments, result, traceback and routing details. Filter by task type and worker, or search with Flower's query syntax, like kwargs:user_id=42 or name:invoice AND state:FAILURE.
• Actions:
  • Retry a task: it's sent again with the arguments Celery recorded, and you check them first.
  • Revoke a waiting task, terminate a running one, or remove a finished one from the board.
  • Send any task.
  • Revoke all or clean all the tasks you're looking at.
• Workers: status, load, pool and queues, plus their active, reserved and scheduled tasks and registered tasks. Restart, grow or shrink the pool, start or stop consuming from a queue, or shut a worker down.
• Queues: messages waiting in the broker (Redis or RabbitMQ), the workers consuming each queue, a warning for queues nobody consumes, and purge.
• Periodic tasks: your beat_schedule, with each task's next run and a button to run it now.
• Settings: light and dark themes in Flower's green, adjustable refresh, relative or absolute times, and local or UTC.
• Operations: read-only mode, basic auth, and Prometheus metrics at /metrics.
• No build step: Preact + htm, and everything ships with the package, fonts included, so it works offline.

Install

pip install celery-board                # runs on Tornado, like Flower
pip install "celery-board[starlette]"   # to mount it in Starlette or FastAPI, or --server starlette
pip install "celery-board[aiohttp]"     # to add it to aiohttp, or --server aiohttp

Python 3.10 or newer. Celery Board depends on flower>=2.2,<2.3, because it uses Flower's internals.

Run it

celery -A proj board
celery -A proj board --port 8080 --url-prefix /board --basic-auth admin:secret
celery -A proj board --server starlette     # or aiohttp

Then open http://localhost:5555. python -m celery_board -A proj board does the same.

The options work like Flower's options of the same name. Each one can also come from a CELERY_BOARD_* environment variable, like CELERY_BOARD_BASIC_AUTH.

Or mount it in your app

Each adapter takes your Celery app and the same options as keyword arguments, like read_only=True or basic_auth="admin:secret". You can also pass it a celery_board.Board.

Starlette / FastAPI

from celery_board.starlette import create_app

app.mount("/board", create_app(celery_app))

Starlette doesn't run a mounted app's lifespan, so the board starts capturing events with its first request. To start it along with your app, run its lifespan from yours:

board = create_app(celery_app)

@asynccontextmanager
async def lifespan(app):
    async with board.router.lifespan_context(board):
        yield

app = FastAPI(lifespan=lifespan)
app.mount("/board", board)

aiohttp

from celery_board.aiohttp import create_app

app.add_subapp("/board", create_app(celery_app))

Tornado

from celery_board.tornado import create_app, handlers

create_app(celery_app, url_prefix="/board").listen(5555)
# or, in your own app:
Application(handlers(celery_app, "/board") + your_handlers)

Tornado has no startup hook, so the board starts with its first request. You can also call Board.start() once the IOLoop runs.

Serve the board from one process. Each process captures events and keeps its own copy of the tasks.

Security

• Basic auth: --basic-auth (or basic_auth=) protects the page and the API. As in Flower, /healthcheck and /metrics stay open for probes and Prometheus.
• Your app's auth: mounted in your app, the board also sits behind your app's authentication.
• Read-only: --read-only hides every action, and the API refuses them.
• Cross-site writes: the board refuses writes from other sites (it checks Sec-Fetch-Site and Origin, like Flower), so a page elsewhere can't act through a logged-in browser.
• Framing: pages can't be framed by other sites.
• Task data: arguments, results and tracebacks show as your workers report them. To keep secrets out, send tasks with Celery's argsrepr and kwargsrepr.

How it works

Celery Board depends on Flower and uses its core as it is:

• flower.events captures worker events in a thread. It keeps tasks and workers in memory, up to max_tasks, updates the Prometheus metrics, and with persistent saves them to a file.
• flower.inspector asks workers for their stats, queues and task lists.
• flower.utils.search indexes tasks for filtering, search and counts.
• flower.utils.broker reads queue lengths from Redis or RabbitMQ.

On top of that:

• celery_board.board holds the dashboard's queries and actions.
• celery_board.web answers HTTP requests without depending on a framework.
• The adapters: celery_board.tornado, .starlette and .aiohttp only translate requests and responses.

Good to know:

• The board knows the tasks it has seen since it started, or since its last save with --persistent. It keeps the latest max_tasks.
• Retry sends a new task with the arguments Celery recorded in its events. Those are reprs: cut at 1024 characters, and replaced by argsrepr when a task sets one. So the board shows them as JSON for you to check before sending. When they aren't plain values, you fill them in.
• Worker details, like queues and task lists, refresh every minute, and every few seconds on a worker's page.

API

Coming from Flower

• The same core: Celery Board runs Flower's own event capture and inspection, so it sees your cluster the way Flower does.
• The same port and options: it takes Flower's port and most of its options, and it serves the same flower_* Prometheus metrics.
• Not included: Flower's OAuth login (put the board behind your own auth or proxy), and Flower's REST API paths, which the board's API above replaces. Autoscale, rate limits and time limits aren't in the UI.

Development

python -m venv .venv && .venv/bin/pip install -e ".[dev]"
redis-server --daemonize yes
.venv/bin/pytest                    # the end-to-end test uses redis://localhost:6379/15, or CELERY_BOARD_TEST_REDIS
.venv/bin/python examples/demo.py   # two workers, the board and some traffic, at http://localhost:5555

Credits

Built by rkwap. Celery Board stands on the shoulders of:

• Flower by Mher Movsisyan and contributors (BSD-3-Clause): the event capture, inspection, search and metrics the board runs on, and the colors it uses.
• bull-board by felixmosh (MIT): the dashboard design it follows.
• Preact (MIT) and htm (Apache-2.0) by Jason Miller, and Feather icons by Cole Bemis (MIT), all bundled. See THIRD_PARTY_NOTICES.md.

Celery Board is not affiliated with or endorsed by the Flower or Celery projects.

License

BSD 3-Clause, like Celery and Flower.