Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

A simple logger for multiple Python processes.

Project Description

A simple, multiprocess-safe logger for Python

Why

Python’s built-in loggers are pretty handy - they’re easily customized and come with useful functionality out of the box, including things like file rotation. These file handlers are thread-safe, but not process-safe, so, if you’re running a webserver in a pre-forking environment, for example, you run the risk of your workers trampling over each other when writing to a common log file. File locking is a possible workaround, but that’s yucky.

To avoid this, it is recommended that one uses a socket-based logger (a code sample is helpfully provided in the Logging Cookbook). However, it is just a code snippet. Multilog is a dependency-free implementation of the sample socket logger with some niceties, like fileConfig support, and parameterization.

How

Once installed, the Multilog daemon can be invoked via:

mutlilog

Usage:

usage: multilog [-h] [-s SERVER] [-p PORT] [-c CONFIG_PATH]

A simple logger for multiple Python processes.

optional arguments:
  -h, --help                    show this help message and exit
  -s SERVER, --server SERVER
                                The server hostname (default: localhost)
  -p PORT, --port PORT          The port to listen on. (default: 9020)
  -c CONFIG_PATH, --config CONFIG_PATH
                                The log configuration to load. (default: logging.ini)

By default, it will look for a logging.ini file in the current directory. If one isn’t found, Multilog will yell at you. A sample configuration file for the server:

[loggers]
keys=root

[handlers]
keys=multilogServerHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=NOTSET
handlers=multilogServerHandler

[handler_multilogServerHandler]
class=handlers.TimedRotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('/var/log/appName/appName.log', 'midnight')

[formatter_simpleFormatter]
class=logging.Formatter
format=%(asctime)s %(levelname)7s: PID: %(process)5s | %(message)s [in %(pathname)s:%(lineno)d]

and for your application:

[loggers]
keys=root

[handlers]
keys=multilogClientHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=NOTSET
handlers=multilogClientHandler

[handler_multilogClientHandler]
class=handlers.SocketHandler
level=DEBUG
formatter=simpleFormatter
args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)

[formatter_simpleFormatter]
class=logging.Formatter
format=%(asctime)s %(levelname)7s: PID: %(process)5s | %(message)s [in %(pathname)s:%(lineno)d]

The important field is the args block in the handler_multilogClientHandler section - those parameters should correspond to the server and ports on which the multilog daemon is listening. By default, the daemon uses localhost and logging.handlers.DEFAULT_TCP_LOGGING_PORT.

For Power Users

If you want to have Multilog share your application’s config, you can do the following:

[loggers]
keys=root,appName

[handlers]
keys=multilogClientHandler,multilogServerHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=NOTSET
handlers=%(root_handler)s

[logger_appName]
level=INFO
handlers=
propagate=1
qualname=appName

[handler_multilogClientHandler]
class=handlers.SocketHandler
level=DEBUG
formatter=simpleFormatter
args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)

[handler_multilogServerHandler]
class=handlers.TimedRotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('/var/log/appName/appName.log', 'midnight')

[formatter_simpleFormatter]
class=logging.Formatter
format=%(asctime)s %(levelname)7s: PID: %(process)5s | %(message)s [in %(pathname)s:%(lineno)d]

Then, in your application, pass the root handler name into the logging config:

import logging
logging.config.fileConfig(config_path, defaults={"root_handler": "multilogClientHandler"})

Multilog will always load the multilogServerHandler handler. If you don’t want to run Multilog (if you’re running a single-threaded local dev server, for example), simply change your root_handler value to multilogServerHandler to write to the handler.

Support

Multilog is compatible with Python 2.6, 2.7, and 3.3+.

Release History

Release History

This version
History Node

1.1.0

History Node

1.0.0

History Node

0.9.0

History Node

0.9.0dev

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
multilog-1.1.0.tar.gz (5.3 kB) Copy SHA256 Checksum SHA256 Source Feb 8, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting