celery_zabbix
Sends task execution metrics to Zabbix: how many tasks were started and have completed successfully or with failure, and how many tasks are still in the queues (supported only for broker redis). Tested against Zabbix-3.0.
Inspired by https://gitlab.com/kalibrr/celery-prometheus
Usage
Run bin/celery zabbix --zabbix-nodename myhost.example.com --zabbix-server zabbix.example.com. (Alternatively you can pass --zabbix-agent-config=/etc/zabbix/zabbix_agentd.conf, then the values for server+nodename will be read from there.)
Import the corresponding Zabbix Template to set up the matching items.
Items
The following items will be sent every 60 seconds (pass --dump-interval=x to configure):
celery.task.started
celery.task.succeeded (with trigger > {$WARN_TASKS_FAILED})
celery.task.failed
celery.task.retried
These are counted from the time the monitoring process started, so you’ll need to process them as delta on the Zabbix server.
celery.task.queuetime (only if task_send_sent_event is enabled)
celery.task.runtime
These are the median values and use the item type “Numeric (float)”.
Queue lengths
If you pass --queuelength-interval=x then every x seconds the queue lengths will be checked (NOTE: this only works with redis as the broker), and the following items will also be sent:
celery.queue[myqueuename]
These are gauge values, i.e. they contain the length as it was retrieved each time, so they can go up and down.
celery.discover.queues
This is an auto-discovery item, together with the Zabbix Template this will create an item and trigger for each queue. On your host, create macros of the form {$WARN_QUEUE_LENGTH:myqueuename} to determine the trigger threshold.
Run tests
Using tox and py.test. Maybe install tox (e.g. via pip install tox) and then simply run tox.
celery_zabbix changes
1.1.0 (2019-11-28)
Make Python-3 compatible. You’ll need a version of zbxsend that is py3 compatible as well, e.g. from https://github.com/pistolero/zbxsend/pull/9
1.0.4 (2019-06-26)
Allow using agent config file and still overriding values on the commandline (Patch by Peter Radcliffe)
1.0.3 (2018-03-23)
Close redis connection we use for queue length checking after use
Catch exceptions in the dump_stats and check_queue_length threads
1.0.2 (2018-03-21)
Be defensive about missing task.runtime
1.0.1 (2018-03-09)
Fix typo in agent config parsing
1.0.0 (2018-03-06)
Initial release
Metadata
Release files for celery-zabbix 1.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| celery_zabbix-1.1.0.tar.gz | 8.7 kB | Details |
Release files / celery_zabbix-1.1.0.tar.gz
| Download URL | celery_zabbix-1.1.0.tar.gz |
|---|---|
| Size | 8.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
19cf6c7de8ddd59834ced689a6cf84bbd322daabb3bab12a2a24fe94a9b57e98
|
|
BLAKE2b-256 checksum How to use checksums |
e1fee2652ef4291eaee635f94f4f9dd78a45e5de4e47410a131879b01c30d423
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.7.5
|