Skip to main content

celery_zabbix

Sends task execution metrics to Zabbix: how many tasks were started and have completed successfully or with failure, and how many tasks are still in the queues (supported only for broker redis). Tested against Zabbix-3.0.

Inspired by https://gitlab.com/kalibrr/celery-prometheus

Usage

  • Run bin/celery zabbix --zabbix-nodename myhost.example.com --zabbix-server zabbix.example.com. (Alternatively you can pass --zabbix-agent-config=/etc/zabbix/zabbix_agentd.conf, then the values for server+nodename will be read from there.)

  • Import the corresponding Zabbix Template to set up the matching items.

Items

The following items will be sent every 60 seconds (pass --dump-interval=x to configure):

  • celery.task.started

  • celery.task.succeeded (with trigger > {$WARN_TASKS_FAILED})

  • celery.task.failed

  • celery.task.retried

These are counted from the time the monitoring process started, so you’ll need to process them as delta on the Zabbix server.

  • celery.task.queuetime (only if task_send_sent_event is enabled)

  • celery.task.runtime

These are the median values and use the item type “Numeric (float)”.

Queue lengths

If you pass --queuelength-interval=x then every x seconds the queue lengths will be checked (NOTE: this only works with redis as the broker), and the following items will also be sent:

  • celery.queue[myqueuename]

These are gauge values, i.e. they contain the length as it was retrieved each time, so they can go up and down.

  • celery.discover.queues

This is an auto-discovery item, together with the Zabbix Template this will create an item and trigger for each queue. On your host, create macros of the form {$WARN_QUEUE_LENGTH:myqueuename} to determine the trigger threshold.

Run tests

Using tox and py.test. Maybe install tox (e.g. via pip install tox) and then simply run tox.

celery_zabbix changes

1.1.0 (2019-11-28)

1.0.4 (2019-06-26)

  • Allow using agent config file and still overriding values on the commandline (Patch by Peter Radcliffe)

1.0.3 (2018-03-23)

  • Close redis connection we use for queue length checking after use

  • Catch exceptions in the dump_stats and check_queue_length threads

1.0.2 (2018-03-21)

  • Be defensive about missing task.runtime

1.0.1 (2018-03-09)

  • Fix typo in agent config parsing

1.0.0 (2018-03-06)

  • Initial release

Metadata

Release files for celery-zabbix 1.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for celery-zabbix 1.1.0
File Size Uploaded
celery_zabbix-1.1.0.tar.gz 8.7 kB Details

Release files / celery_zabbix-1.1.0.tar.gz

Download URL celery_zabbix-1.1.0.tar.gz
Size 8.7 kB
Tags Source
SHA-256 checksum
How to use checksums
19cf6c7de8ddd59834ced689a6cf84bbd322daabb3bab12a2a24fe94a9b57e98
BLAKE2b-256 checksum
How to use checksums
e1fee2652ef4291eaee635f94f4f9dd78a45e5de4e47410a131879b01c30d423
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.5

Release history Release notifications | RSS feed

This release

1.1.0 This release

1 release file

1.0.4

1 release file

1.0.3

1 release file

1.0.2

1 release file

1.0.1

1 release file

1.0.0

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page