Skip to main content
Help us improve PyPI by participating in user testing. All experience levels needed!

Sends task execution metrics to Zabbix

Project description

Sends task execution metrics to Zabbix: how many tasks were started and have completed successfully or with failure, and how many tasks are still in the queues (supported only for broker redis). Tested against Zabbix-3.0.

Inspired by https://gitlab.com/kalibrr/celery-prometheus

Usage

  • Run bin/celery zabbix --zabbix-nodename myhost.example.com --zabbix-server zabbix.example.com. (Alternatively you can pass --zabix-agent-config=/etc/zabbix/zabbix_agentd.conf, then the values for server+nodename will be read from there.)
  • Import the corresponding Zabbix Template to set up the matching items.

Items

The following items will be sent every 60 seconds (pass --dump-interval=x to configure):

  • celery.task.started
  • celery.task.succeeded (with trigger > {$WARN_TASKS_FAILED})
  • celery.task.failed
  • celery.task.retried

These are counted from the time the monitoring process started, so you’ll need to process them as delta on the Zabbix server.

  • celery.task.queuetime (only if task_send_sent_event is enabled)
  • celery.task.runtime

These are the median values and use the item type “Numeric (float)”.

Queue lengths

If you pass --queuelength-interval=x then every x seconds the queue lengths will be checked (NOTE: this only works with redis as the broker), and the following items will also be sent:

  • celery.queue[myqueuename]

These are gauge values, i.e. they contain the length as it was retrieved each time, so they can go up and down.

  • celery.discover.queues

This is an auto-discovery item, together with the Zabbix Template this will create an item and trigger for each queue. On your host, create macros of the form {$WARN_QUEUE_LENGTH:myqueuename} to determine the trigger threshold.

Run tests

Using tox and py.test. Maybe install tox (e.g. via pip install tox) and then simply run tox.

celery_zabbix changes

1.0.3 (2018-03-23)

  • Close redis connection we use for queue length checking after use
  • Catch exceptions in the dump_stats and check_queue_length threads

1.0.2 (2018-03-21)

  • Be defensive about missing task.runtime

1.0.1 (2018-03-09)

  • Fix typo in agent config parsing

1.0.0 (2018-03-06)

  • Initial release

Project details


Release history Release notifications

This version
History Node

1.0.3

History Node

1.0.2

History Node

1.0.1

History Node

1.0.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
celery_zabbix-1.0.3.tar.gz (8.3 kB) Copy SHA256 hash SHA256 Source None Mar 23, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page