tangods-graylogger 1.0.0

A tango device server to catch logs on the old log4tango API and send them to Graylogger using GELF and HTTP.

The Graylogger Tango Device Server

A Tango device server to catch log messages generated by other device servers, using the traditional log4tango implementation, and redirect them to Graylog using GELF HTTP.

Repository content

• Graylogger tangoDS: a minimalistic tango device that can be used as logging target. This tangoDS outputs to Graylog HTTP, using graylog GELF format. To test, set the logging_target property of the log producer to log to device::my/gray/logger, where my/gray/logger is the tango name of your Graylogger tangoDS.
  In the Graylogger device, you must set the hostname and port properties to point to your Graylog server.
  In graylog you must set up an input that processes GELF HTTP requests. The default port is 12201.

• Producer tangoDS: a software tango device that has commands for producing log messages using the various *_stream methods (self.debug_stream(), self.error_stream() etc...).
  It also has a wrapper for exceptions that catches a generic exception, alters state and status, sends to self.error_stream() and re-raises them.

• A docker compose file to spin a graylog container. To start it, start systemd docker.service then navigate to the docker directory and run docker-compose -f docker-compose.yml. Stop with <Ctlr-C>. The messages and settings are stored on disk.
  To reverto to a blank installation run docker-compose down -v when not running. To log in, go to localhost:9000 and login as admin password password12345.

• A docker configuration file: config/docker.conf. This is mostly to tune the container performance.

Observations and annotations

Graylog was chosen because it is very convenient for log analysis. The GELF protocol was chosen for its simplicity (only a few items are mandatory) and because it is possible to extend it with other tags, the user defined tags simply start with undescore. Http was chosen because it's simple to implement and because the server replies to post request, making it possible to understand if the log was sent succesfully. It is also possible to switch to https for enhanced security, even if that is not implemented yet.

A good place for tango logging documentation is the RFC.

Tango logging has room for improvement. The amount of information and format is limited and not all features are implemented. Exceptions are not added to the logs automatically, not even if changing the logging_level: changes to the code are necessary. The easiest way to do so in python is to use wrappers. See the example in Producer.

The methods that are named debug_stream() and similar are not really streams, they are loggers.

If you start a tango device with -v3 or -v5 the logging behavior changes completely: logging_target and logging_level are overridden. -v5 adds extra corba logs to the stream. The corba logs are not documented (AFAIK).

It is not entirely possible to set the logging target in jive, as multi stream requires a multi line property that is not possible to input in jive. One way to do so, is by using pytango Database() class.

itango example: Database().put_device_property('my/log/producer', {"logging_target": ['console', 'file::/home/me/test.log', 'device::my/gray/logger']}).

This adds 3 logging targets, one to console, one to file and one to another tango device.

If the logging target tangoDS is restarterd, the whole logging will stop. It is not possible to initialize the devices in order, they have to be restarted, because the logging is set in a stage before the init itself. The tango device which act as a log target needs to be started first, then all the other devices can start. There is no way around this and this is an important limitation. It could be worth to look into file based approaches.

To work around these limitations, the Graylogger tango device server has commands that allow it to reset the logging mechanism on the other devices.

Sardana seems to do things differently, it does not seem to respond to set_logging_target() and set_logging_level(). It uses standard python logging. I was unable to use a device or console target.