Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Stream supervisor events to a logstash instance

Project Description

This is a port of the Supervisor-logging project. Along with capturing loglines, as Supervisor-logging does, it’s also intended to capture the PROCESS_STATE events that Supervisor emits.

supervisor-logstash-notifier

A supervisor plugin to stream events & logs to a Logstash instance.

Installation

Python 2.6, 2.7 or Python 3.2+ is required.

pip install supervisor-logstash-notifier

Note that Supervisor itself does not yet work on Python 3, though it can be installed in a separate environment (because supervisor-logstash-notifier is a separate process).

Usage

The Logstash instance to send the events to is configured with the environment variables:

  • LOGSTASH_SERVER
  • LOGSTASH_PORT
  • LOGSTASH_PROTO

Add the plugin as an event listener:

# Capture state changes
[eventlistener:logging]
command = logstash_notifier
events = PROCESS_STATE

# Capture stdout/stderr
[eventlistener:logging]
command = logstash_notifier
events = PROCESS_LOG

# Capture state changes and stdout/stderr
[eventlistener:logging]
command = logstash_notifier
events = PROCESS_STATE,PROCESS_LOG

If you don’t wish to define the environment variables for the entire shell, you can pass them in via Supervisor’s configuration:

[eventlistener:logging]
environment=LOGSTASH_SERVER="127.0.0.1",LOGSTASH_PORT="12202",LOGSTASH_PROTO="tcp"
command=logstash_notifier
events=PROCESS_STATE

Advanced Usage

It is also possible to include environment variables in the event messages, by specifying the name of the environment variables to include:

[eventlistener:logging]
command=export IPV4=`ec2metadata --local-ipv4`; logstash_notifier --include IPV4
events=PROCESS_STATE

Or alternatively, by specifying arbitrary keyvals of data to log:

[eventlistener:logging]
command=logstash_notifier --include bears="polar,brown,black" notbears="unicorn,griffin,sphinx,otter"
events=PROCESS_STATE

These two forms of arbitrary user data inclusion can be combined, and used together if necessary.

Running with Logstash

Logstash can be simply configured to receive events:

input {
    tcp {
        port => 12201
        codec => json
    }
}

output {
    stdout {
        codec => rubydebug
    }
}

The JSON produced by the events and log output will look like this:

# State changes
{
  "@timestamp": "2016-03-28T23:58:03.469Z",
  "@version": "1",
  "eventname": "PROCESS_STATE_STOPPED",
  "from_state": "STOPPING",
  "groupname": "myprocess",
  "host": "ip-10-93-130-24",
  "level": "INFO",
  "logger_name": "supervisor",
  "message": "PROCESS_STATE_STOPPED collectd",
  "path": "/path/to/supervisor-logstash-notifier/logstash_notifier/__init__.py",
  "pid": "1234",
  "processname": "myprocess",
  "tags": [],
  "type": "logstash"
}

# Log output
{
  "@timestamp": "2016-03-28T23:58:03.741Z",
  "@version": "1",
  "channel": "stdout"
  "eventname": "PROCESS_LOG_STDOUT",
  "groupname": "myprocess",
  "host": "localhost",
  "level": "INFO",
  "logger_name": "supervisor",
  "message": "myprocess output #1\n",
  "path": "/path/to/supervisor-logstash-notifier/logstash_notifier/__init__.py",
  "pid": "1234",
  "processname": "myprocess",
  "tags": [],
  "type": "logstash",
}
Release History

Release History

This version
History Node

0.2.4

History Node

0.2.3

History Node

0.2.2

History Node

0.2.1

History Node

0.2.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
supervisor_logstash_notifier-0.2.4-py2.py3-none-any.whl (8.6 kB) Copy SHA256 Checksum SHA256 py2.py3 Wheel Sep 22, 2016
supervisor-logstash-notifier-0.2.4.tar.gz (6.1 kB) Copy SHA256 Checksum SHA256 Source Sep 22, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting