AWS CloudWatch Fluent Metrics

Project description

# FluentMetrics ## First Things First Please make sure to review the [current AWS CloudWatch Custom Metrics pricing]( https://aws.amazon.com/cloudwatch/pricing/) before proceeding. ## Overview FluentMetrics is an easy-to-use Python module that makes logging CloudWatch custom metrics a breeze. The goal is to provide a framework for logging detailed metrics with a minimal footprint. When you look at your code logic, you want to see your actual code logic, not line after line of metrics logging. FluentMetrics lets you maximize your metrics footprint while minimizing your metrics code footprint. ## Installation You can install directly from PyPI: sh pip install fluentmetrics  ## 'Fluent' . . . what is that? Fluent describes an easy-to-read programming style. The goal of fluent development is to make code easier to read and reduce the amount of code required to build objects. It's easier to take a look a comparison between fluent and non-fluent style. #### Non-Fluent Example sh g = Game() f = Frame(Name='Tom') f.add_score(7) f.add_score(3) g.add_frame(f) f = Frame(Name='Tom') f.add_strike() g.add_frame(f)  #### Non-Fluent Example with Constructor sh g = Game() g.add_frame(Frame(Name='Tom', Score1=7, Score2=3) g.add_frame(Frame(Name='Tom', Score1=10)  #### Fluent Example sh g = Game() g.add_frame(Frame().with_name('Tom').score(3).spare()) g.add_frame(Frame().with_name('Tom').strike())  While the difference may seem to be nitpicking, a frame is really just a constructed object. In the first example, we're taking up three lines of code to create the object--there's nothing wrong with that. However, in the second example, we're using constructors. This is slightly more readable, but there's a great deal of logic bulked up in our constructor. In the third example, we're using fluent-style code as it starts at creating the frame and *fluently* continues until it's created the entire frame in a single line. And more importantly, *it's readable.* We're not just creating an object with a massive constructor or spending several lines of code just to create a single object. ## Terminology Quickstart #### Namespaces Every metric needs to live in a namespace. Since you are logging your own custom metrics, you need to provide a custom namespace for your metric. Click [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-namespaces.html) for a list of the standard AWS namespaces. *Example*: In this example, we're creating a simple FluentMetric in a namespace called Performance. This means that every time we log a metric with m, we will automatically log it to the Performance namespace. sh m = FluentMetric().with_namespace('Performance')  #### Metric Names The metric name is the thing you are actually logging. Each value that you log must be tied to a metric name. When you log a custom metric with a new metric name, the name will automatically be created if it doesn't already exist. Click [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/viewing_metrics_with_cloudwatch.html) to see existing metrics that can help you define names for your custom metrics. *Example*: In this example, we're logging two metrics called StartupTime and StuffTime to the Performance namespace (we only needed to define the namespace once). sh m = FluentMetric().with_namespace('Performance') m.log(MetricName='StartupTime', Value=27, Unit='Seconds') do_stuff() m.log(MetricName='StuffTime', Value=12000, Unit='Milliseconds')  #### Values Obviously we need to log a value with each metric. This needs to be a number since we convert this value to a float before sending to CloudWatch. **IMPORTANT**: When logging multiple values for the same custom metric within a minute, CloudWatch aggregates an average over a minute. Click [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#publishingDataPoints) for more details. #### Dimensions A dimension defines how you want to slice and dice the metric. These are simply name-value pairs and you can define up to 10 per metric. Click [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html#usingDimensions) for more details on using dimensions. **IMPORTANT:** When you define multiple dimensions, CloudMetrics attaches all of those dimensions to the metric as a single combined dimension set--think of them as an aggregate primary key. For example, if you log a metric with the dimensions os = 'linux' and flavor='ubunutu' you will only be able to aggregate by **both** os and flavor. You **cannot** aggregate only by just os or just flavor. FluentMetrics solves this problem by automatically logging three metrics--one for os, one for flavor and then one for the combied dimensions, giving you maximum flexibility. *Example*: In this example, we're logging boot/restart time metrics. When this code executes, we will end up with 6 metrics: * BootTime and RestartTime for os * BootTime and RestartTime for instance-id * BootTime and RestartTime for 'os and instance-id sh m = FluentMetric().with_namespace('Performance/EC2') \ .with_dimension('os', 'linux'). \ .with_dimension('instance-id', 'i-123456') boot_time = start_instance() m.log(MetricName='BootTime', Value=boot_time, Unit='Milliseconds') restart_time = restart_instance() m.log(MetricName='RestartTime', Value=restart_time, Unit='Milliseconds')  #### Units CloudWatch has built-in logic to provide meaning to the metric values. We're not just logging a value--we're looking a value of some unit. By defining the unit type, CloudWatch will know how to properly present, aggregate and compare that value with other values. For example, if you submit a value with unit Milliseconds, then it can properly aggregate it up to seconds, minutes or hours. This is a list of the most current valid list of units. A more up-to-date list should be available [here](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html) under the **Unit** section,. sh "Seconds"|"Microseconds"|"Milliseconds"|"Bytes"|"Kilobytes"|"Megabytes"| "Gigabytes"|"Terabytes"|"Bits"|"Kilobits"|"Megabits"|"Gigabits"|"Terabits"| "Percent"|"Count"|"Bytes/Second"|"Kilobytes/Second"|"Megabytes/Second"| "Gigabytes/Second"|"Terabytes/Second"|"Bits/Second"|"Kilobits/Second"| "Megabits/Second"|"Gigabits/Second"|"Terabits/Second"|"Count/Second"|"None"  ##### Unit Shortcut Methods If you don't want to type out the individual unit name, there are shortcut methods for each unit. sh m = FluentMetric().with_namespace('Performance/EC2') \ .with_dimension('os', 'linux'). \ .with_dimension('instance-id', 'i-123456') m.seconds(MetricName='CompletionInSeconds', Value='1000') m.microseconds(MetricName='CompletionInMicroseconds', Value='1000') m.milliseconds(MetricName='CompletionInMilliseconds', Value='1000') m.bytes(MetricName='SizeInBytes', Value='1000') m.kb(MetricName='SizeInKb', Value='1000') m.mb(MetricName='SizeInMb', Value='1000') m.gb(MetricName='SizeInGb', Value='1000') m.tb(MetricName='SizeInTb', Value='1000') m.bits(MetricName='SizeInBits', Value='1000') m.kbits(MetricName='SizeInKilobits', Value='1000') m.mbits(MetricName='SizeInMegabits', Value='1000') m.gbits(MetricName='SizeInGigabits', Value='1000') m.tbits(MetricName='SizeInTerabits', Value='1000') m.pct(MetricName='Percent', Value='20') m.count(MetricName='ItemCount', Value='20') m.bsec(MetricName='BandwidthBytesPerSecond', Value='1000') m.kbsec(MetricName='BandwidthKilobytesPerSecond', Value='1000') m.mbsec(MetricName='BandwidthMegabytesPerSecond', Value='1000') m.gbsec(MetricName='BandwidthGigabytesPerSecond', Value='1000') m.tbsec(MetricName='BandwidthTerabytesPerSecond', Value='1000') m.bitsec(MetricName='BandwidthBitsPerSecond', Value='1000') m.kbitsec(MetricName='BandwidthKilobitsPerSecond', Value='1000') m.mbitsec(MetricName='BandwidthMegabitsPerSecond', Value='1000') m.gbitsec(MetricName='BandwidthGigabitsPerSecond', Value='1000') m.tbitsec(MetricName='BandwidthTerabitsPerSecond', Value='1000') m.countsec(MetricName='ItemCountsPerSecond', Value='1000')  #### Timers One of the most common uses of logging is measuring performance. FluentMetrics allows you to activate multiple built-in timers by name and log the elapsed time in a single line of code. **NOTE:** The elapsed time value is automatically stored as unit Milliseconds. *Example*: In this example, we're starting timers workflow and job1 at the same time. Timers start as soon as you create them and never stop running. When you call elapsed, FluentMetrics will log the number of elapsed milliseconds with the MetricName. sh m = FluentMetric() m.with_timer('workflow').with_timer('job1') do_job1() m.elapsed(MetricName='Job1CompletionTime', JobTimerName='job1') m.with_timer('job2') do_job2() m.elapsed(MetricName='Job2CompletionTime', JobTimerName='job2') finish_workflow() m.elapsed(MetricName='WorkflowCompletionTime', JobTimerName='workflow')  #### Metric Stream ID A key feature of FluentMetrics is the metric stream ID. This ID will be added as a dimension and logged with every metric. The benefit of this dimension is to provide a distinct stream of metrics for an end-to-end operation. When you create a new instance of FluentMetric, you can either pass in your own value or FluentMetrics will generate a GUID. In CloudWatch, you can then see all of the metrics for a particular stream ID in chronological order. A metric stream can be a job, or a server or any way that you want to unique group a contiguous stream of metrics. *Example*: In this example, we'll have two metrics in the Performance namespace, each with metric stream ID of abc-123. We can then go to CloudWatch and filter by that stream ID to see the entire operation performance at a glance. sh m = FluentMetric().with_namespace('Performance').with_stream_id('abc-123') m.log(MetricName='StartupTime', Value=100, Unit='Seconds') do_work() m.log(MetricName='WorkCompleted', Value=1000, Unit='Milliseconds')  ## Use Case Quickstart #### #1: Least Amount of Code Required to Log a Metric This is the minimal amount of work you need to log--create a FluentMetric with a namespace, then log a value. **Result**: This code will log a single value 100 for ActiveServerCount in the Stats namespace. sh from fluentmetrics.metric import FluentMetric m = FluentMetric().with_namespace('Stats') m.log(MetricName='ActiveServerCount', Value='100', Unit='Count')  #### #2: Logging Multiple Metrics to the Same Namespace If you are logging multiple metrics to the same namespace, this is a great use case for FluentMetrics. You only need to create one instance of FluentMetric and specify a different metric name when you call log. **Result**: This code will log a single value 100 for ActiveServerCount in the Stats namespace. sh from fluentmetrics.metric import FluentMetric m = FluentMetric().with_namespace('Stats') m.log(MetricName='ActiveServerCount', Value='10', Unit='Count') \ .log(MetricName='StoppedServerCount', Value='20', Unit='Count') \ .log(MetricName='ActiveLinuxCount', Value='50', Unit='Count') \ .log(MetricName='ActiveWindowsCount', Value='50', Unit='Count')  #### #3: Logging Counts In the previous example, we logged a metric and identified the unit Count. Instead of specifying the unit, you can specify the type of object **Result**: This code will log a single value 100 for ActiveServerCount in the Stats namespace. sh from fluentmetrics.metric import FluentMetric m = FluentMetric().with_namespace('Stats') m.count(MetricName='ActiveServerCount', Value='10') 

Release history Release notifications | RSS feed

Uploaded source