Zibrato provides Python decorators and context managers that instrument code.
It attempts to do this very efficiently, so the metrics are collected and
then shifted into a ZeroMQ queue, where a backend process can collect them
and send them to Librato. Other backends such as statsd are contemplated for
Zibrato is available through PyPi at http://pypi.python.org/pypi/Zibrato/
Alternatively, one should be able to clone the github repository at
https://github.com/Version2beta/zibrato and run pip install the
Please note that pyzmq is an installation prerequisite. Ubuntu users (and
probably others) will need the python-dev package installed in order to build
sudo apt-get install python-dev
To run tests, you'll also need Python nose and Gary Bernhardt's expecter
pip install nose expecter
Zibrato consists of two parts. First, it provides the zibrato module used for
from zibrato import Zibrato
Librato, Backend, and Broker modules also available. See code.
z = Zibrato()
Set up a new instance of Zibrato to use in your code. Accepts several
* host: The FQDN or IP address with which to connect. Optional.
Defaults to '127.0.0.1'. See "Zibrato workers" below.
* port: The port with which to connect. Optional. Defaults to 5550.
* context: A ZeroMQ context instance. This is completely optional and
only desirable under advanced circumstances where ZeroMQ is being
used in other ways, too.
@z.count_me(level = 'info', name = 'counter_name')
Increment a counter named 'counter_name' each time the decorated function
z.Count_me(level = 'info', name = 'counter_name', value = 4)
Increment a counter named 'counter_name'. To increment by an interval other
than one, set value to the amount.
@z.time_me(level = 'debug', name = 'timer_name')
Record the decorated function's execution time under a gauge named
z.Time_me(level = 'debug', name = 'timer_name')
Record the time spent within a given context.
z.gauge(level = 'crit', name = 'gauge_name', value=123)
Record a value.
Zibrato decorators, of course, return the result of the wrapped function.
Context managers return None as they are not intended to be used from within
the 'with' block. The gauge method returns None also.
Metric decorators and context managers take up to four arguments:
* level: Required. Monitoring level, modeled after logging levels (i.e.
debug, info, warning, error, critical) but completely arbitrary so you
can use whatever labels work for you. Zibrato workers are configured to
pay attention to only specified levels.
* mtype: Required. Type of metric, typically 'counter', 'timer', and
'gauge'. See "Metric types" below.
* name: Required. Name of the metric being recorded.
* value: Value to record. For timing functions, value is neither required
nor desirable to provide, and if provided it will be replaced by the
measured time. For counters, value represents the quantity by which the
counter should be incremented, and defaults to 1 if not provided. For
gauges, this is the fixed reading and should be provided.
* source: The source of the metric. This might represent the name of the
program, class, or server for instance. Optional and defaults to
z = Zibrato()
# or z = Zibrato(host = '127.0.0.1', port = '5550')
@z.time_me(level = 'debug', name = 'myfunct_timer', source = 'myprog')
@z.count_me(level = 'info', name = "myfunct_counter', value = 5) # inc by 5
with z.Count_me(level = 'info', name = 'counter_name', source = 'deathstar'):
with z.Time_me(level = 'debug', name = 'timer_name'):
z.gauge(level = 'crit', name = 'gauge_name', value=123)
Zibrato requires a broker that connects one or more publishers
of measurements (source code being run in parallel) to one or
more measurement backends (Librato and Statsd, for example.)
The Zibrato broker
The Zibrato broker runs as a daemon under supervisord or other process
controller. It provides a TCP endpoint for the Zibrato publishers (code
instruments) to send measurements, and a TCP endpoint to which the Zibrato
backends can subscribe to get measurements for sending off to other services.
The broker might be started like this::
/usr/bin/librato-broker --host 127.0.0.1 --port 5550
where 'host' specifies the IP address or FQDN and 'port' specifies the lower
port of a consecutive pair to which the broker should bind. Both host and port
are optional. Default values are 127.0.0.1 and 5550, respectively. The lower
port address is used to listen to Zibrato publishers (see 'Zibrato module'
above) and the higher port is used to publish to Zibrato workers (see 'Zibrato
The preferred way to start the Zibrato backend would be to use a service such
command=/usr/bin/zibrato-broker --host 127.0.0.1 --port 5550
This version of Zibrato also includes a worker that processes queued
measurements and sends them to Librato.
python /usr/bin/zibrato-librato --username USERNAME --apitoken KEY \
--levels test,debug,info --flush 60
The available parameters are:
* --host: The FQDN hostname or IP address of the Zibrato backend.
* --port: The port to which the Zibrato work should connect. This is the
higher of the two ports in the pair, and one greater than the port
specified when starting the backend.
* --levels: The levels to which this worker should subscribe.
* --flush: The frequency with which the measurements should be sent to
* --username: The Librato username for connecting to their API.
* --apitoken: The Librato API Token for connecting to their API.
Alternatively, the worker can be run from supervisord::
command=python /usr/bin/zibrato-librato --username USERNAME \
--apitoken KEY --levels info,warn --flush 60
Creating a new Zibrato worker
New Zibrato backend workers should subclass the Backend class specified in
zibrato/backend.py. They probably need to reimplement the connect, parse,
post, and flush methods, and must include code for running as __main__. See
zibrato/librato.py as an example.
The setup.py script contains an "entry_points" section for defining console
scripts. The preferred implementation of an additional worker will be to name
it following the "zibrato-service" pattern and add it to the console_scripts
* Counters. Zibrato counters keep track of how many times an event with
a common name happens between two flushes on the back end. So for
example, let's say you're keeping track of how may times 'myfunct' is
called, and you're flushing your data to the back end every 60 seconds.
If you don't specify a value, then the 'myfunct_counter' will be
incremented by one each time the counter is encountered, sent to Librato
and reset to zero every 60 seconds. If source is specified, the counter
uniquely tracked by source and name, rather than just name. The
timestamp for a counter is given as the time the counter is flushed.
* Gauges. Gauges hold a value at a given time. Each gauge measurement
is recorded to the backend with a timestamp for the time Zibrato
received the measurement.
* Timers. Zibrato provides a special gauge that it fills in automatically
with the amount of time something took. Time is measured in seconds to
microsecond resolution using Python's datetime.now() method.
Please note that the Zibrato backend is ultimately responsible for
implementing how each metric type is recorded. In this release only one
backend is provided, but in future releases check with the backend
documentation to determing exactly how a metric behaves.
Please also note that Zibrato was originally written to provide code
instrumentation specifically and to connect to Librato specifically. This
introduces an impedence mismatch, as Librato's availble metric types as of
this writing are limited to gauges and counters, and Librato's counters
don't work the way we need our counters to work. So the metrics implemented
in this code translate into only gauges at Librato.
Zibrato includes nose tests in the tests/ directory.
Pull requests are welcome!
Special thanks to Tracy Harms @kaleidic who coached me on Agile methodologies
and test driven development, plus helped tease out the intricasies of ZeroMQ
and the architecture of this program.
Thanks too to Gary Bernhardt @garybernhardt for the expecter library, and for
helping Tracy and I figure out how to translate Ruby TDD experience into
Also, thank you to regulars on #zeromq who answered beginner questions
Zibrato is released under a 3-clause BSD license, which can be read in the
TODO: Brief introduction on what you do with files - including link to relevant help section.