A simple logger for multiple Python processes.
Project description
A simple, multiprocess-safe logger for Python
Why
Python’s built-in loggers are pretty handy - they’re easily customized and come with useful functionality out of the box, including things like file rotation. These file handlers are thread-safe, but not process-safe, so, if you’re running a webserver in a pre-forking environment, for example, you run the risk of your workers trampling over each other when writing to a common log file. File locking is a possible workaround, but that’s yucky.
To avoid this, it is recommended that one uses a socket-based logger (a code sample is helpfully provided in the Logging Cookbook). However, it is just a code snippet. Multilog is a dependency-free implementation of the sample socket logger with some niceties, like fileConfig support, and parameterization.
How
Once installed, the Multilog daemon can be invoked via:
mutlilog
Usage:
usage: multilog [-h] [-s SERVER] [-p PORT] [-c CONFIG_PATH]
A simple logger for multiple Python processes.
optional arguments:
-h, --help show this help message and exit
-s SERVER, --server SERVER
The server hostname (default: localhost)
-p PORT, --port PORT The port to listen on. (default: 9020)
-c CONFIG_PATH, --config CONFIG_PATH
The log configuration to load. (default: logging.ini)
By default, it will look for a logging.ini file in the current directory. If one isn’t found, Multilog will yell at you. A sample configuration file for the server:
[loggers]
keys=root
[handlers]
keys=multilogServerHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=NOTSET
handlers=multilogServerHandler
[handler_multilogServerHandler]
class=handlers.TimedRotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('/var/log/appName/appName.log', 'midnight')
[formatter_simpleFormatter]
class=logging.Formatter
format=%(asctime)s %(levelname)7s: PID: %(process)5s | %(message)s [in %(pathname)s:%(lineno)d]
and for your application:
[loggers]
keys=root
[handlers]
keys=multilogClientHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=NOTSET
handlers=multilogClientHandler
[handler_multilogClientHandler]
class=handlers.SocketHandler
level=DEBUG
formatter=simpleFormatter
args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)
[formatter_simpleFormatter]
class=logging.Formatter
format=%(asctime)s %(levelname)7s: PID: %(process)5s | %(message)s [in %(pathname)s:%(lineno)d]
The important field is the args block in the handler_multilogClientHandler section - those parameters should correspond to the server and ports on which the multilog daemon is listening. By default, the daemon uses localhost and logging.handlers.DEFAULT_TCP_LOGGING_PORT.
For Power Users
If you want to have Multilog share your application’s config, you can do the following:
[loggers]
keys=root,appName
[handlers]
keys=multilogClientHandler,multilogServerHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=NOTSET
handlers=%(root_handler)s
[logger_appName]
level=INFO
handlers=
propagate=1
qualname=appName
[handler_multilogClientHandler]
class=handlers.SocketHandler
level=DEBUG
formatter=simpleFormatter
args=('localhost', handlers.DEFAULT_TCP_LOGGING_PORT)
[handler_multilogServerHandler]
class=handlers.TimedRotatingFileHandler
level=DEBUG
formatter=simpleFormatter
args=('/var/log/appName/appName.log', 'midnight')
[formatter_simpleFormatter]
class=logging.Formatter
format=%(asctime)s %(levelname)7s: PID: %(process)5s | %(message)s [in %(pathname)s:%(lineno)d]
Then, in your application, pass the root handler name into the logging config:
import logging
logging.config.fileConfig(config_path, defaults={"root_handler": "multilogClientHandler"})
Multilog will always load the multilogServerHandler handler. If you don’t want to run Multilog (if you’re running a single-threaded local dev server, for example), simply change your root_handler value to multilogServerHandler to write to the handler.
Support
Multilog is compatible with Python 2.6, 2.7, and 3.3+.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file multilog-1.1.0.tar.gz
.
File metadata
- Download URL: multilog-1.1.0.tar.gz
- Upload date:
- Size: 5.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3e1774fea21f8e06cb72ceab8c4c0f457d21a41342d1f2d10bd41a1668f05991 |
|
MD5 | 890c9312bdfc8ce61a835cb4150fa9c7 |
|
BLAKE2b-256 | 3c2e609b8488313cefa6c3ce757357155de7be1014d823224a88a2ffba32f725 |