markus 4.0.0

Metrics system for generating statistics about your app

Markus is a Python library for generating metrics.

Code:

https://github.com/willkg/markus

Issues:

https://github.com/willkg/markus/issues

License:

MPL v2

Documentation:

http://markus.readthedocs.io/en/latest/

Goals

Markus makes it easier to generate metrics in your program by:

• providing multiple backends (Datadog statsd, statsd, logging, logging rollup, and so on) for sending data to different places

• sending metrics to multiple backends at the same time

• providing a testing framework for easy testing

• providing a decoupled architecture making it easier to write code to generate metrics without having to worry about making sure creating and configuring a metrics client has been done–similar to the Python logging Python logging module in this way

I use it at Mozilla in the collector of our crash ingestion pipeline. Peter used it to build our symbols lookup server, too.

Install

To install Markus, run:

$ pip install markus

(Optional) To install the requirements for the markus.backends.datadog.DatadogMetrics backend:

$ pip install markus[datadog]

Quick start

Similar to using the logging library, every Python module can create a markus.main.MetricsInterface (loosely equivalent to a Python logging logger) at any time including at module import time and use that to generate metrics.

For example:

import markus

metrics = markus.get_metrics(__name__)

Creating a markus.main.MetricsInterface using __name__ will cause it to generate all stats keys with a prefix determined from __name__ which is a dotted Python path to that module.

Then you can use the markus.main.MetricsInterface anywhere in that module:

@metrics.timer_decorator("chopping_vegetables")
def some_long_function(vegetable):
    for veg in vegetable:
        chop_vegetable()
        metrics.incr("vegetable", value=1)

At application startup, configure Markus with the backends you want to use to publish metrics and any options they require.

For example, lets configure metrics to publish to logs and Datadog:

import markus

markus.configure(
    backends=[
        {
            # Log metrics to the logs
            "class": "markus.backends.logging.LoggingMetrics",
        },
        {
            # Log metrics to Datadog
            "class": "markus.backends.datadog.DatadogMetrics",
            "options": {
                "statsd_host": "example.com",
                "statsd_port": 8125,
                "statsd_namespace": ""
            }
        }
    ]
)

When you’re writing your tests, use the markus.testing.MetricsMock to make testing easier:

from markus.testing import MetricsMock


def test_something():
    with MetricsMock() as mm:
        # ... Do things that might publish metrics

        # Make assertions on metrics published
        mm.assert_incr_once("some.key", value=1)

History

4.0.0 (October 22nd, 2021)

Features

• Added support for Python 3.10 (#88)

Backwards incompatibel changes

• Dropped support for Python 3.6 (#89)

3.0.0 (February 5th, 2021)

Features

• Added support for Python 3.9 (#79). Thank you, Brady!

• Changed assert_* helper methods on markus.testing.MetricsMock to print the records to stdout if the assertion fails. This can save some time debugging failing tests. (#74)

Backwards incompatible changes

• Dropped support for Python 3.5 (#78). Thank you, Brady!

• markus.testing.MetricsMock.get_records and markus.testing.MetricsMock.filter_records return markus.main.MetricsRecord instances now. This might require you to rewrite/update tests that use the MetricsMock.

2.2.0 (April 15th, 2020)

Features

• Add assert_ methods to MetricsMock to reduce the boilerplate for testing. Thank you, John! (#68)

Bug fixes

• Remove use of six library. (#69)

2.1.0 (October 7th, 2019)

Features

• Fix get_metrics() so you can call it without passing in a thing and it’ll now create a MetricsInterface that doesn’t have a key prefix. (#59)

2.0.0 (September 19th, 2019)

Features

• Use time.perf_counter() if available. Thank you, Mike! (#34)

• Support Python 3.7 officially.

• Add filters for adjusting and dropping metrics getting emitted. See documentation for more details. (#40)

Backwards incompatible changes

• tags now defaults to [] instead of None which may affect some expected test output.

• Adjust internals to run .emit() on backends. If you wrote your own backend, you may need to adjust it.

• Drop support for Python 3.4. (#39)

• Drop support for Python 2.7.

  If you’re still using Python 2.7, you’ll need to pin to <2.0.0. (#42)

Bug fixes

• Document feature support in backends. (#47)

• Fix MetricsMock.has_record() example. Thank you, John!

1.2.0 (April 27th, 2018)

Features

• Add .clear() to MetricsMock making it easier to build a pytest fixture with the MetricsMock context and manipulate records for easy testing. (#29)

Bug fixes

• Update Cloudwatch backend fixing .timing() and .histogram() to send histogram metrics type which Datadog now supports. (#31)

1.1.2 (April 5th, 2018)

Typo fixes

• Fix the date from the previous release. Ugh.

1.1.1 (April 5th, 2018)

Features

• Official switch to semver.

Bug fixes

• Fix MetricsMock so it continues to work even if configure is called. (#27)

1.1 (November 13th, 2017)

Features

• Added markus.utils.generate_tag utility function

1.0 (October 30th, 2017)

Features

• Added support for Python 2.7.

• Added a markus.backends.statsd.StatsdMetrics backend that uses pystatsd client for statsd pings. Thank you, Javier!

Bug fixes

• Added LoggingRollupMetrics to docs.

• Mozilla has been running Markus in production for 6 months so we can mark it production-ready now.

0.2 (April 19th, 2017)

Features

• Added a markus.backends.logging.LoggingRollupMetrics backend that rolls up metrics and does some light math on them. Possibly helpful for light profiling for development.

Bug fixes

• Lots of documentation fixes. Thank you, Peter!

0.1 (April 10th, 2017)

Initial writing.