talisker 0.9.0

A common WSGI stack

Talisker is a runtime for your wsgi app that aims to provide a common platform for your python services.

tl;dr

Simply run your wsgi app with talisker as if it was gunicorn.:

talisker app:wsgi -c config.py ...

Talisker will wrap your app in a some simple WSGI middleware, and configure logging to output structured logging like so:

logger = logging.getLogger('app')
logger.info('something happened', extra={'context': 'I haz it'})

will output:

2016-01-13 10:24:07.357Z INFO app "something happened" svc.context="I haz it" request_id=...

It also exposes some status endpoints you can use, go to the /_status/ url on your app to see them.

This all works out of the box by using the talisker runner instead of gunicorn’s, and there are many more features you can use too.

Elevator Pitch

Talisker is based on a number of standard python tools:

• stdlib logging for logs

• gunicorn for a wsgi runner

• requests for http requests

• statsd for metrics (and optionally, prometheus_client)

• sentry for errors

• werkzeug for thread locals and wsgi utilities

It also supports additionaly optional tools:

• celery for async tasks

• prometheus as an alternate metrics tool

It’s main job is to integrate and configure all the above in a single tool, for use in both dev and production, which provides a standard set of features out of the box:

• drop-in replacement for gunicorn as a wsgi runner

• standardised structured logging on top of python stdlib logging

• request id tracing

• standard set of status endpoints for your app

• easier configuration via env vars

• metrics for everything

• deep sentry integration (WIP)

All the above are available by just using the talisker entry point script, rather than gunicorn.

In addition, with a small amount of effort, your app can benefit from additional features:

• add structured logging tags to your application logs

• simple deeper nagios checks - just implement a /_status/check url in your app

• per-thread requests connection pool management

Additionally, talisker provides additional tools for integrating with your infrastructure:

• grok filters for log parsing

• rsyslog templates and config for log shipping (TODO)

Talisker is opinionated, and derived directly from the authors’ needs and as such not currently very configurable. However, PR’s are very welcome!

For more information, see The Documentation, which should be found at:

https://talisker.readthedocs.io

0.9.0 (2017-01-24)

The major feature in this release is support for sentry, which is integrated with wsgi, logging, and celery. Also supports opt-in integration with flask and django, see the relevant docs for more info.

Other changes

• refactor of how logging contexts were implemented. More flexible and reliable. Note talisker.logs.extra_logging and talisker.logs.set_logging_context are now deprecated, you should use talisker.logs.logging_context and talisker.logs.logging_context.push, respectively, as covered in the updated logging docs.

• improved celery logging, tasks logs now have task_id and task_name automatically added to their logs.

• improved logging messages when parsing TALISKER_NETWORKS at startup

0.8.0 (2016-12-13)

• prometheus: add optinal support for promethues_client

• celery: request id automatically sent and logged, and support for 4.0

• docs: initial ‘talisker contract’

• statsd: better client initialisation

• internal: refactoring of global variables, better /_status/ url dispatch

0.7.1 (2016-11-09)

• remove use of future’s import hooks, as they mess with raven’s vendored imports

• slight tweak to logfmt serialisation, and update docs to match

0.7.0 (2016-11-03)

Upgrading

This release includes a couple of minor backwards incompatible changes:

1. access logs now use the talisker format, rather than CLF. See the docs for more info. If you are using access logs already, then the easiest upgrade path is to output the access logs to stderr (access_logfile=”-“), and delete your old log files.

2. talisker no longer prefixes developer supplied tags with ‘svc.’. This should only matter if you’ve already set up dashboards or similar with the old prefixed name, and you will need to remove the prefix

Changes:

• access logs now in logfmt rather than CLF

• dummy statsd client is now useful in testing

• logs are colored in development, to aid reading

• the ‘svc’ prefix for tags has been removed

0.6.7 (2016-10-05)

• actually include the encoding fix for check endpoint

0.6.6 (2016-10-05)

• add celery metrics

• fix issue with encoding in check endpoint when iterable

0.6.5 (2016-09-26)

• make celery runner actually work, wrt logging

0.6.4 (2016-09-23)

• fix encoding issue with X-Request-Id header (again!)

0.6.3 (2016-09-21)

• fix setuptools entry points, which were typoed into oblivion.

0.6.2 (2016-09-21)

• make gunicorn use proper statsd client

• log some extra warnings if we try to configure gunicorn things that talisker overides.

• better documented public api via __all__

• first take on some celery helpers

• some packaging improvements

0.6.1 (2016-09-12)

• actually do remove old DEBUGLOG backups, as backupCount=0 does not remove any. Of course.

0.6.0 (2016-09-09)

• Propagate gunicorn.error log, and remove its default handler.

This allows consistant logging, making the choice in all cases that your gunicorn logs go to the same stream as your other application log, making the choice in all cases that your gunicorn logs go to the same stream as your other application logs.

We issue a warning if the user tries to configure errorlog manually, as it won’t work as expected.