Skip to main content

Helper for pushing AWS CloudWatch metrics to Graphite

Project description

https://travis-ci.org/crccheck/cloudwatch-to-graphite.svg

Cloudwatch-to-Graphite (leadbutt) is a small utility to take metrics from CloudWatch to Graphite.

Installation

Install using pip:

pip install cloudwatch-to-graphite

Configuring boto

Cloudwatch-to-Graphite uses boto, so make sure to follow its configuration instructions. The easiest way to do this is to set up the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

Usage

Configuration Files

If you have a simple setup, the easiest way to get started is to set up a config.yaml. You can copy the included config.yaml.example. Then just run:

leadbutt

If you have several configs you want to switch between, you can specify a custom configuration file:

leadbutt --config-file=production.yaml -n 20

You can even generate configs on the fly and send them in via stdin by setting the config file to ‘-‘:

generate_config_from_inventory | leadbutt --config-file=-

There’s a helper to generate configuration files called plumbum. Use it like:

plumbum [-r REGION] [-f FILTER] [--token TOKEN] template namespace

Namespace is the CloudWatch namespace for the resources of interest; for example AWS/RDS. The template is a Jinja2 template. You can add arbitrary replacement tokens, eg {{ replace_me }}, and then pass in values on the CLI via --token. For example, if you called:

plumbum --token replace_me='hello, world' sample_templates/rds.yml.j2 AWS/RDS

You would get all instances of {{ replace_me }} in the templace replaced with hello, world.

Filters

You can pass simple key=value filters in to plumbum; be aware of the limitations:

  • the filters run against whatever the AWS API has returned; if you have a lot of objects of whatever type, expect the API request to take a while.

  • they work only against object attributes and tags returned by the API. For example, RDS and ELB objects can be tagged, but as getting the tags is a per-object subrequest; plumbum does not do those, so you can only filter on the object attributes.

Example: plumbum -f Name=my-dev-instance sample_templates/ec2.yml.j2 ec2

Sending Data to Graphite

If your graphite server is at graphite.local, you can send metrics by chaining with netcat:

leadbutt | nc -q0 graphite.local 2003

Or if you want to use UDP:

leadbutt | nc -uw0 graphite.local 2003

If you need to namespace your metrics for a hosted Graphite provider, you could provide a custom formatter, but the easiest way is to just run the output through awk:

leadbutt | \
  awk -v namespace="$HOSTEDGRAPHITE_APIKEY" '{print namespace"."$0}' | \
  nc -uw0 my-graphite-provider.xxx 2003

Customizing Your Graphite Metric Names

Set the Formatter option to set the template used to generate Graphite metric names. I wasn’t sure what should be default, so I copied cloudwatch2graphite’s. Here’s what it looks like:

cloudwatch.%(Namespace)s.%(dimension)s.%(MetricName)s.%(statistic)s.%(Unit)s

TitleCased variables come directly from the YAML configuration, while lowercase variables are derived:

  • statistic – the current statistic since Statistics can be a list

  • dimension – the dimension value, e.g. “i-r0b0t” or “my-load-balancer”

The format string is Python’s %-style.

config.yaml

What metrics are pulled is in a YAML configuration file. See the example config.yaml.example for an idea of what you can do.

Developing

See: : Contributing.

Useful References

Prior Art

Cloudwatch-to-Graphite was inspired by edasque’s cloudwatch2graphite. I was looking to expand it, but I wanted to use boto.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cloudwatch-to-graphite-0.11.0.tar.gz (8.7 kB view details)

Uploaded Source

Built Distribution

cloudwatch_to_graphite-0.11.0-py2.py3-none-any.whl (12.4 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file cloudwatch-to-graphite-0.11.0.tar.gz.

File metadata

File hashes

Hashes for cloudwatch-to-graphite-0.11.0.tar.gz
Algorithm Hash digest
SHA256 cdf1a5b9718b610f123e853e633ae22d472ce8494528c48e60b5210b86a949d7
MD5 903cec0da2f4a9e1a5166ad44aae4e3c
BLAKE2b-256 f0c44ea77f6c25b9e110b3c5d0f7b559784ef7c4099e01aeeed119bb9207cde8

See more details on using hashes here.

File details

Details for the file cloudwatch_to_graphite-0.11.0-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for cloudwatch_to_graphite-0.11.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 99b91d392dbabbfb540ee570049eced299da6f2a8e1306d77413f0550469f044
MD5 ead3e6fc4ac0b62b73163c81767cb5fe
BLAKE2b-256 a56a8d5b196378c7a4890bd1a2b6ff7a9985a5843188102f8259afebe7e55656

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page