Skip to main content

Metrics system for generating statistics about your app

Project description

Markus is a Python library for generating metrics.

Code:https://github.com/willkg/markus
Issues:https://github.com/willkg/markus/issues
License:MPL v2
Documentation:http://markus.readthedocs.io/en/latest/

Goals

Markus makes it easier to generate metrics in your program by:

  • providing multiple backends (Datadog statsd, statsd, logging, logging rollup, and so on) for sending data to different places
  • sending metrics to multiple backends at the same time
  • providing a testing framework for easy testing
  • providing a decoupled architecture making it easier to write code to generate metrics without having to worry about making sure creating and configuring a metrics client has been done–similar to the Python logging Python logging module in this way

I use it at Mozilla in the collector of our crash ingestion pipeline. Peter used it to build our symbols lookup server, too.

Install

To install Markus, run:

$ pip install markus

(Optional) To install the requirements for the markus.backends.datadog.DatadogMetrics backend:

$ pip install markus[datadog]

Quick start

Similar to using the logging library, every Python module can create a MetricsInterface (loosely equivalent to a Python logging logger) at any time including at module import time and use that to generate metrics.

For example:

import markus

metrics = markus.get_metrics(__name__)

Creating a MetricsImplementation using __name__ will cause it to generate all stats keys with a prefix determined from __name__ which is a dotted Python path to that module.

Then you can use the MetricsImplementation anywhere in that module:

@metrics.timer_decorator('chopping_vegetables')
def some_long_function(vegetable):
    for veg in vegetable:
        chop_vegetable()
        metrics.incr('vegetable', value=1)

At application startup, configure Markus with the backends you want to use to publish metrics and any options they require.

For example, lets configure metrics to publish to logs and Datadog:

import markus

markus.configure(
    backends=[
        {
            # Log metrics to the logs
            'class': 'markus.backends.logging.LoggingMetrics',
        },
        {
            # Log metrics to Datadog
            'class': 'markus.backends.datadog.DatadogMetrics',
            'options': {
                'statsd_host': 'example.com',
                'statsd_port': 8125,
                'statsd_namespace': ''
            }
        }
    ]
)

When you’re writing your tests, use the MetricsMock to make testing easier:

import markus
from markus.testing import MetricsMock


def test_something():
    with MetricsMock() as mm:
        # ... Do things that might publish metrics

        # This helps you debug and write your test
        mm.print_records()

        # Make assertions on metrics published
        assert mm.has_record(markus.INCR, 'some.key', value=1)

History

2.1.0 (October 7th, 2019)

Features

  • Fix get_metrics() so you can call it without passing in a thing and it’ll now create a MetricsInterface that doesn’t have a key prefix. (#59)

2.0.0 (September 19th, 2019)

Features

  • Use time.perf_counter() if available. Thank you, Mike! (#34)
  • Support Python 3.7 officially.
  • Add filters for adjusting and dropping metrics getting emitted. See documentation for more details. (#40)

Backwards incompatible changes

  • tags now defaults to [] instead of None which may affect some expected test output.

  • Adjust internals to run .emit() on backends. If you wrote your own backend, you may need to adjust it.

  • Drop support for Python 3.4. (#39)

  • Drop support for Python 2.7.

    If you’re still using Python 2.7, you’ll need to pin to <2.0.0. (#42)

Bug fixes

  • Document feature support in backends. (#47)
  • Fix MetricsMock.has_record() example. Thank you, John!

1.2.0 (April 27th, 2018)

Features

  • Add .clear() to MetricsMock making it easier to build a pytest fixture with the MetricsMock context and manipulate records for easy testing. (#29)

Bug fixes

  • Update Cloudwatch backend fixing .timing() and .histogram() to send histogram metrics type which Datadog now supports. (#31)

1.1.2 (April 5th, 2018)

Typo fixes

  • Fix the date from the previous release. Ugh.

1.1.1 (April 5th, 2018)

Features

  • Official switch to semver.

Bug fixes

  • Fix MetricsMock so it continues to work even if configure is called. (#27)

1.1 (November 13th, 2017)

Features

  • Added markus.utils.generate_tag utility function

1.0 (October 30th, 2017)

Features

  • Added support for Python 2.7.
  • Added a markus.backends.statsd.StatsdMetrics backend that uses pystatsd client for statsd pings. Thank you, Javier!

Bug fixes

  • Added LoggingRollupMetrics to docs.
  • Mozilla has been running Markus in production for 6 months so we can mark it production-ready now.

0.2 (April 19th, 2017)

Features

  • Added a markus.backends.logging.LoggingRollupMetrics backend that rolls up metrics and does some light math on them. Possibly helpful for light profiling for development.

Bug fixes

  • Lots of documentation fixes. Thank you, Peter!

0.1 (April 10th, 2017)

Initial writing.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for markus, version 2.1.0
Filename, size File type Python version Upload date Hashes
Filename, size markus-2.1.0-py3-none-any.whl (23.3 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size markus-2.1.0.tar.gz (32.3 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page