Concurrent logging handler (drop-in replacement for RotatingFileHandler)
This module provides an additional log handler for Python’s standard logging package (PEP 282). This handler will write a log entries to a set of files, which switches from one file to the next when the current file reaches a certain size. Multiple processes can write to the log file concurrently.
The ConcurrentRotatingFileHandler class is a drop-in replacement for Python’s standard log handler RotatingFileHandler. This module uses file locking so that multiple processes can concurrently log to a single file without dropping or clobbering log records. This module provides a file rotation scheme like RotatingFileHanler, except that extra care is taken to ensure that logs can be safely rotated before the rotation process is started. (This module works around the file rename issue with RotatingFileHandler on Windows, where a rotation failure means that all subsequent logs are lost).
This module’s attempts to preserve log records at all cost. This means that log files will grow larger than the specified maximum (rotation) size. So if disk space is tight, you may want to stick with RotatingFileHandler, which will strictly adhere to the maximum file size.
Use the following command to install this package:
If you are installing from source, you can use:
python setup.py install
Here is a example demonstrating how to use this module directly (from within Python code):
from logging import getLogger, INFO from cloghandler import ConcurrentRotatingFileHandler import os log = getLogger() # Use an absolute path to prevent file rotation trouble. logfile = os.path.abspath("mylogfile.log") # Rotate log after reaching 512K, keep 5 old copies. rotateHandler = ConcurrentRotatingFileHandler(logfile, "a", 512*1024, 5) log.addHandler(rotateHandler) log.setLevel(INFO) log.info("Here is a very exciting log message, just for you")
Automatic fallback example
If you are distributing your code and you are unsure if the ConcurrentLogHandler package has been installed everywhere your code will run, Python makes it easy to setup gracefully fall back to the built in RotatingFileHandler, here is an example:
try: from cloghandler import ConcurrentRotatingFileHandler as RFHandler except ImportError: from warnings import warn warn("ConcurrentLogHandler package not installed. Using builtin log handler") from logging.handlers import RotatingFileHandler as RFHandler log = getLogger() rotateHandler = RFHandler("/path/to/mylogfile.log", "a", 1048576, 15) log.addHandler(rotateHandler)
Config file example
This example shows you how to use this log handler with the logging config file parser. This allows you to keep your logging configuration code separate from your application code.
Example config file: logging.ini:
[loggers] keys=root [handlers] keys=hand01 [formatters] keys=form01 [logger_root] level=NOTSET handlers=hand01 [handler_hand01] class=handlers.ConcurrentRotatingFileHandler level=NOTSET formatter=form01 args=("rotating.log", "a", 512*1024, 5) [formatter_form01] format=%(asctime)s %(levelname)s %(message)s
Example Python code: app.py:
import logging, logging.config import cloghandler logging.config.fileConfig("logging.ini") log = logging.getLogger() log.info("Here is a very exciting log message, just for you")
- 0.8.2: Minor bug fix release (again)
- Found and resolved another issue with older logging packages that do not support encoding.
- 0.8.1: Minor bug fix release
- Now importing “codecs” directly; I found some slight differences in the logging module in different Python 2.4.x releases that caused the module to fail to load.
- 0.8.0: Minor feature release
- Add better support for using logging.config.fileConfig(). This class is now available using class=handlers.ConcurrentRotatingFileHandler.
- Minor changes in how the filename parmerer is handled when given a relative path.
- 0.7.4: Minor bug fix
- Fixed a typo in the package description (incorrect class name)
- Added a change log; which you are reading now.
- Fixed the close() method to no longer assume that stream is still open.
- This module has had minimal testing in a multi-threaded process. I see no reason why this should be an issue, but no stress-testing has been done in a threaded situation. If this is important to you, you could always add threading support to the stresstest.py script and send me the patch.
Release history Release notifications
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size & hash SHA256 hash help||File type||Python version||Upload date|
|ConcurrentLogHandler-0.8.2-py2.4.egg (25.6 kB) Copy SHA256 hash SHA256||Egg||2.4|
|ConcurrentLogHandler-0.8.2-py2.5.egg (25.5 kB) Copy SHA256 hash SHA256||Egg||2.5|
|ConcurrentLogHandler-0.8.2.tar.gz (20.6 kB) Copy SHA256 hash SHA256||Source||None|