Skip to main content

Sends task execution metrics to Zabbix

Project description

Sends task execution metrics to Zabbix: how many tasks were started and have completed successfully or with failure, and how many tasks are still in the queues (supported only for broker redis). Tested against Zabbix-3.0.

Inspired by https://gitlab.com/kalibrr/celery-prometheus

Usage

  • Run bin/celery zabbix --zabbix-nodename myhost.example.com --zabbix-server zabbix.example.com. (Alternatively you can pass --zabbix-agent-config=/etc/zabbix/zabbix_agentd.conf, then the values for server+nodename will be read from there.)

  • Import the corresponding Zabbix Template to set up the matching items.

Items

The following items will be sent every 60 seconds (pass --dump-interval=x to configure):

  • celery.task.started

  • celery.task.succeeded (with trigger > {$WARN_TASKS_FAILED})

  • celery.task.failed

  • celery.task.retried

These are counted from the time the monitoring process started, so you’ll need to process them as delta on the Zabbix server.

  • celery.task.queuetime (only if task_send_sent_event is enabled)

  • celery.task.runtime

These are the median values and use the item type “Numeric (float)”.

Queue lengths

If you pass --queuelength-interval=x then every x seconds the queue lengths will be checked (NOTE: this only works with redis as the broker), and the following items will also be sent:

  • celery.queue[myqueuename]

These are gauge values, i.e. they contain the length as it was retrieved each time, so they can go up and down.

  • celery.discover.queues

This is an auto-discovery item, together with the Zabbix Template this will create an item and trigger for each queue. On your host, create macros of the form {$WARN_QUEUE_LENGTH:myqueuename} to determine the trigger threshold.

Run tests

Using tox and py.test. Maybe install tox (e.g. via pip install tox) and then simply run tox.

celery_zabbix changes

1.0.4 (2019-06-26)

  • Allow using agent config file and still overriding values on the commandline (Patch by Peter Radcliffe)

1.0.3 (2018-03-23)

  • Close redis connection we use for queue length checking after use

  • Catch exceptions in the dump_stats and check_queue_length threads

1.0.2 (2018-03-21)

  • Be defensive about missing task.runtime

1.0.1 (2018-03-09)

  • Fix typo in agent config parsing

1.0.0 (2018-03-06)

  • Initial release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

celery_zabbix-1.0.4.tar.gz (8.5 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page