This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

JSON log formatter and Sentry client

Project Description

Utilities for working with [Sentry](https://github.com/getsentry/sentry). We got into trouble using the in-app Sentry client [raven-python](https://github.com/getsentry/raven-python) client with high volumes so we decided to take the HTTP posting to Sentry out of the app itself. Snitch helps here.

log2json

is a stdlib log formatter that will output log record in Sentry’s JSON format. You can use any handler, but we use it with a file handler so the app can always quickly dump the log record and does not have to wait for an HTTP post.

snitch

monitors a logfile for new lines (like tail but skips all existing content) and sends each new line in a HTTP POST request to the Sentry API. Snitch reloads the log file every half minute or so. On one hand, this is a workaround to deal with log rotation, on the other hand, this deals with high log volumes by just dropping everything and resynching with the present. I mean, when it reopens the file it effectively skips everything that was already logged to the file but not yet processed because it continues at the end of the file after a reopen. This is what we want. Also, snitch does a time.sleep(.1) between all POSTS as a rate-limiter.

alternatives

I haven’t looked into using UDP instead of TCP for sending the messages. This may solve problems on the sending host, but still has the potential of flooding Sentry.

Release History

Release History

This version
History Node

0.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
snitch-0.1.tar.gz (5.1 kB) Copy SHA256 Checksum SHA256 Source Jan 2, 2013
snitch-0.2.tar.gz (5.1 kB) Copy SHA256 Checksum SHA256 Source Jan 8, 2013
snitch-0.3.tar.gz (5.2 kB) Copy SHA256 Checksum SHA256 Source Apr 10, 2013

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting