markus 2.0.0

Metrics system for generating statistics about your app

Markus is a Python library for generating metrics.

Code:

https://github.com/willkg/markus

Issues:

https://github.com/willkg/markus/issues

License:

MPL v2

Documentation:

http://markus.readthedocs.io/en/latest/

Goals

Markus makes it easier to generate metrics in your program by:

• providing multiple backends (Datadog statsd, statsd, logging, logging rollup, and so on) for sending data to different places

• sending metrics to multiple backends at the same time

• providing a testing framework for easy testing

• providing a decoupled architecture making it easier to write code to generate metrics without having to worry about making sure creating and configuring a metrics client has been done–similar to the Python logging Python logging module in this way

I use it at Mozilla in the collector of our crash ingestion pipeline. Peter used it to build our symbols lookup server, too.

Install

To install Markus, run:

$ pip install markus

(Optional) To install the requirements for the markus.backends.datadog.DatadogMetrics backend:

$ pip install markus[datadog]

Quick start

Similar to using the logging library, every Python module can create a MetricsInterface (loosely equivalent to a Python logging logger) at any time including at module import time and use that to generate metrics.

For example:

import markus

metrics = markus.get_metrics(__name__)

Creating a MetricsImplementation using __name__ will cause it to generate all stats keys with a prefix determined from __name__ which is a dotted Python path to that module.

Then you can use the MetricsImplementation anywhere in that module:

@metrics.timer_decorator('chopping_vegetables')
def some_long_function(vegetable):
    for veg in vegetable:
        chop_vegetable()
        metrics.incr('vegetable', value=1)

At application startup, configure Markus with the backends you want to use to publish metrics and any options they require.

For example, lets configure metrics to publish to logs and Datadog:

import markus

markus.configure(
    backends=[
        {
            # Log metrics to the logs
            'class': 'markus.backends.logging.LoggingMetrics',
        },
        {
            # Log metrics to Datadog
            'class': 'markus.backends.datadog.DatadogMetrics',
            'options': {
                'statsd_host': 'example.com',
                'statsd_port': 8125,
                'statsd_namespace': ''
            }
        }
    ]
)

When you’re writing your tests, use the MetricsMock to make testing easier:

import markus
from markus.testing import MetricsMock


def test_something():
    with MetricsMock() as mm:
        # ... Do things that might publish metrics

        # This helps you debug and write your test
        mm.print_records()

        # Make assertions on metrics published
        assert mm.has_record(markus.INCR, 'some.key', value=1)

History

2.0.0 (September 19th, 2019)

Features

• Use time.perf_counter() if available. Thank you, Mike! (#34)

• Support Python 3.7 officially.

• Add filters for adjusting and dropping metrics getting emitted. See documentation for more details. (#40)

Backwards incompatible changes

• tags now defaults to [] instead of None which may affect some expected test output.

• Adjust internals to run .emit() on backends. If you wrote your own backend, you may need to adjust it.

• Drop support for Python 3.4. (#39)

• Drop support for Python 2.7.

  If you’re still using Python 2.7, you’ll need to pin to <2.0.0. (#42)

Bug fixes

• Document feature support in backends. (#47)

• Fix MetricsMock.has_record() example. (Thanks John!)

1.2.0 (April 27th, 2018)

Features

• Add .clear() to MetricsMock making it easier to build a pytest fixture with the MetricsMock context and manipulate records for easy testing. (#29)

Bug fixes

• Update Cloudwatch backend fixing .timing() and .histogram() to send histogram metrics type which Datadog now supports. (#31)

1.1.2 (April 5th, 2018)

Typo fixes

• Fix the date from the previous release. Ugh.

1.1.1 (April 5th, 2018)

Features

• Official switch to semver.

Bug fixes

• Fix MetricsMock so it continues to work even if configure is called. (#27)

1.1 (November 13th, 2017)

Features

• Added markus.utils.generate_tag utility function

1.0 (October 30th, 2017)

Features

• Added support for Python 2.7.

• Added a markus.backends.statsd.StatsdMetrics backend that uses pystatsd client for statsd pings. Thank you, Javier!

Bug fixes

• Added LoggingRollupMetrics to docs.

• Mozilla has been running Markus in production for 6 months so we can mark it production-ready now.

0.2 (April 19th, 2017)

Features

• Added a markus.backends.logging.LoggingRollupMetrics backend that rolls up metrics and does some light math on them. Possibly helpful for light profiling for development.

Bug fixes

• Lots of documentation fixes. Thank you, Peter!

0.1 (April 10th, 2017)

Initial writing.