sprockets-statsd 1.0.0

Asynchronously send metrics to a statsd instance.

Asynchronously send metrics to a statsd instance.

This library provides connectors to send metrics to a statsd instance using either TCP or UDP.

import asyncio
import time

import sprockets_statsd.statsd

statsd = sprockets_statsd.statsd.Connector(
   host=os.environ.get('STATSD_HOST', '127.0.0.1'))

async def do_stuff():
   start = time.time()
   response = make_some_http_call()
   statsd.timing(f'timers.http.something.{response.code}',
                 (time.time() - start))

async def main():
   await statsd.start()
   try:
      do_stuff()
   finally:
      await statsd.stop()

The Connector instance maintains a resilient connection to the target StatsD instance, formats the metric data into payloads, and sends them to the StatsD target. It defaults to using TCP as the transport but will use UDP if the ip_protocol keyword is set to socket.IPPROTO_UDP. The Connector.start method starts a background asyncio.Task that is responsible for maintaining the connection. The timing method enqueues a timing metric to send and the task consumes the internal queue when it is connected.

The following convenience methods are available. You can also call inject_metric for complete control over the payload.

incr	Increment a counter metric
decr	Decrement a counter metric
gauge	Adjust or set a gauge metric
timer	Append a duration to a timer metric using a context manager
timing	Append a duration to a timer metric

If you are a python-statsd user, then the method names should look very familiar. That is quite intentional. I like the interface and many others do as well. There is one very very important difference though – the timing method takes the duration as the number of seconds as a float instead of the number of milliseconds.

Warning

If you are accustomed to using python-statsd, be aware that the timing method expects the number of seconds as a float instead of the number of milliseconds.

Tornado helpers

The sprockets_statsd.tornado module contains mix-in classes that make reporting metrics from your tornado web application simple. You will need to install the sprockets_statsd[tornado] extra to ensure that the Tornado requirements for this library are met.

import asyncio
import logging

from tornado import ioloop, web

import sprockets_statsd.tornado


class MyHandler(sprockets_statsd.tornado.RequestHandler,
                web.RequestHandler):
    async def get(self):
        with self.execution_timer('some-operation'):
            await self.do_something()
        self.set_status(204)

    async def do_something(self):
        await asyncio.sleep(1)


class Application(sprockets_statsd.tornado.Application, web.Application):
    def __init__(self, **settings):
        settings['statsd'] = {
            'host': os.environ['STATSD_HOST'],
            'prefix': 'applications.my-service',
        }
        super().__init__([web.url('/', MyHandler)], **settings)

    async def on_start(self):
        await self.start_statsd()

    async def on_stop(self):
        await self.stop_statsd()


if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)
    app = Application()
    app.listen(8888)
    iol = ioloop.IOLoop.current()
    try:
        iol.add_callback(app.on_start)
        iol.start()
    except KeyboardInterrupt:
        iol.add_future(asyncio.ensure_future(app.on_stop()),
                       lambda f: iol.stop())
        iol.start()

This application will emit two timing metrics each time that the endpoint is invoked:

applications.my-service.timers.some-operation:1001.3449192047119|ms
applications.my-service.timers.MyHandler.GET.204:1002.4960041046143|ms

You will need to set the $STATSD_HOST environment variable to enable the statsd processing inside of the application. The RequestHandler class exposes methods that send counter and timing metrics to a statsd server. The connection is managed by the Application provided that you call the start_statsd method during application startup.

Metrics are sent by a asyncio.Task that is started by start_statsd. The request handler methods insert the metric data onto a asyncio.Queue that the task reads from. Metric data remains on the queue when the task is not connected to the server and will be sent in the order received when the task establishes the server connection.

Integration with sprockets.http

If you use sprockets.http in your application stack, then the Tornado integration will detect it and install the initialization and shutdown hooks for you. The application will just work provided that the $STATSD_HOST and $STATSD_PREFIX environment variables are set appropriately. The following snippet will produce the same result as the Tornado example even without setting the prefix:

class Application(sprockets_statsd.tornado.Application,
                  sprockets.http.app.Application):
    def __init__(self, **settings):
        statsd = settings.setdefault('statsd', {})
        statsd.setdefault('host', os.environ['STATSD_HOST'])
        statsd.setdefault('protocol', 'tcp')
        settings.update({
            'service': 'my-service',
            'environment': os.environ.get('ENVIRONMENT', 'development'),
            'statsd': statsd,
            'version': getattr(__package__, 'version'),
        })
        super().__init__([web.url('/', MyHandler)], **settings)

if __name__ == '__main__':
    sprockets.http.run(Application, log_config=...)

Definint the service and environment in settings as above will result in the prefix being set to:

applications.{self.settings["service"]}.{self.settings["environment"]}

The recommended usage is to:

1. define service, environment, and version in the settings

2. explicitly set the host and protocol settings in self.settings["statsd"]