talisker 0.8.0

A common WSGI stack

Talisker is a runtime for your wsgi app that aims to provide a common platform for your python services.

tl;dr

Simply run your wsgi app with talisker as if it was gunicorn.:

talisker app:wsgi -c config.py …

Talisker will wrap your app in a some simple WSGI middleware, and configure logging to output structured logging like so:

logger = logging.getLogger('app')
logger.info('something happened', extra={'context': 'I haz it'})

will output:

2016-01-13 10:24:07.357Z INFO app "something happened" svc.context="I haz it" request_id=...

It also exposes some status endpoints you can use, go to the /_status/ url on your app to see them.

This all works out of the box by using the talisker runner instead of gunicorn’s, and there are many more features you can use too.

Elevator Pitch

Talisker is based on a number of standard python tools:

• stdlib logging for logs

• gunicorn for a wsgi runner

• requests for http requests

• statsd for metrics (and optionally, prometheus_client)

• raven for errors

• werkzeug for thread locals and wsgi utilities

It is designed specifically to be used in both development and production, and aims to provide a default set of features out of the box:

• drop-in replacement for gunicorn

• standard log format, including ISO/UTC timestamps

• structured logging with python stdlib

• improved gunicorn access logs, with ms precision UTC timestamps

• request id tracing

• standard set of status endpoints for your app

• easier statsd endpoint configuration

• automatic prometheus metrics endpoint (optional)

• sentry/raven middleware (TODO)

All the above are available by just using the talisker entry point script, rather than gunicorn.

In addition, with a small amount of effort, your app can benefit from additional features:

• simple deeper nagios checks - just implement a _status/check url in your app

• per-thread requests connection pool managment (WIP)

• automatic statsd metrics for outgoing HTTP requests (WIP)

• more efficient statsd client management (WIP)

Additionally, talisker provides additional tools for integrating with your infrastructure:

• grok filters for log parsing (WIP)

• rsyslog templates and config for log shipping (TODO)

Talisker is opinionated, and derived directly from the authors’ needs and as such not currently very configurable. However, PR’s are very welcome!

For more information, see The Documentation, which should be found at:

https://talisker.readthedocs.io

0.8.0 (2016-12-13)

• prometheus: add optinal support for promethues_client

• celery: request id automatically sent and logged, and support for 4.0

• docs: initial ‘talisker contract’

• statsd: better client initialisation

• internal: refactoring of global variables, better /_status/ url dispatch

0.7.1 (2016-11-09)

• remove use of future’s import hooks, as they mess with raven’s vendored imports

• slight tweak to logfmt serialisation, and update docs to match

0.7.0 (2016-11-03)

Upgrading

This release includes a couple of minor backwards incompatible changes:

1. access logs now use the talisker format, rather than CLF. See the docs for more info. If you are using access logs already, then the easiest upgrade path is to output the access logs to stderr (access_logfile=”-“), and delete your old log files.

2. talisker no longer prefixes developer supplied tags with ‘svc.’. This should only matter if you’ve already set up dashboards or similar with the old prefixed name, and you will need to remove the prefix

Changes:

• access logs now in logfmt rather than CLF

• dummy statsd client is now useful in testing

• logs are colored in development, to aid reading

• the ‘svc’ prefix for tags has been removed

0.6.7 (2016-10-05)

• actually include the encoding fix for check endpoint

0.6.6 (2016-10-05)

• add celery metrics

• fix issue with encoding in check endpoint when iterable

0.6.5 (2016-09-26)

• make celery runner actually work, wrt logging

0.6.4 (2016-09-23)

• fix encoding issue with X-Request-Id header (again!)

0.6.3 (2016-09-21)

• fix setuptools entry points, which were typoed into oblivion.

0.6.2 (2016-09-21)

• make gunicorn use proper statsd client

• log some extra warnings if we try to configure gunicorn things that talisker overides.

• better documented public api via __all__

• first take on some celery helpers

• some packaging improvements

0.6.1 (2016-09-12)

• actually do remove old DEBUGLOG backups, as backupCount=0 does not remove any. Of course.

0.6.0 (2016-09-09)

• Propagate gunicorn.error log, and remove its default handler.

This allows consistant logging, making the choice in all cases that your gunicorn logs go to the same stream as your other application log, making the choice in all cases that your gunicorn logs go to the same stream as your other application logs.

We issue a warning if the user tries to configure errorlog manually, as it won’t work as expected.

0.5.7 (2016-09-02)

• Update publishing workflow

• Add make changelog target

0.5.6 (2016-09-02)

• more testing release process in prepartion for 0.6

0.5.5 (2016-09-02)

• testing release process in prepartion for 0.6

0.5.4 (2016-08-10)

• series of point release to fix various small bugs

0.5.0 (2016-08-10)

• add grok filters for logstash

• slight adjustment to logfmt serialisation: talisker now strips “ from tag values. This is due to a limitation in logstash.

0.4.1 (2016-08-05)

• publish separate py2/py3 wheels, due to dependency differences

• some doc changes

0.4.0 (2016-08-05)

• First public release an PyPI.