Skip to main content

Logger for multiprocessing applications

Project description

multiprocessing-logging

Run Status Coverage Badge Supported Python versions License

When using the multiprocessing module, logging becomes less useful since sub-processes should log to individual files/streams or there's the risk of records becoming garbled.

This simple module implements a Handler that when set on the root Logger will handle tunneling the records to the main process so that they are handled correctly.

It's currently tested in Linux and Python 2.7 & 3.5+.

Pypy3 hangs on the tests so I don't recommend using it.

Pypy appears to be working, recently.

It was tested by users and reported to work on Windows with Python 3.5 and 3.6.

Origin

This library was taken verbatim from a StackOverflow post and extracted into a module so that I wouldn't have to copy the code in every project.

Later, several improvements have been contributed.

Usage

Before you start logging but after you configure the logging framework (maybe with logging.basicConfig(...)), do the following:

import multiprocessing_logging

multiprocessing_logging.install_mp_handler()

and that's it.

With multiprocessing.Pool

When using a Pool, make sure install_mp_handler is called before the Pool is instantiated, for example:

import logging
from multiprocessing import Pool
from multiprocessing_logging import install_mp_handler

loggig.basicConfig(...)
install_mp_handler()
pool = Pool(...)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for multiprocessing-logging, version 0.3.0
Filename, size File type Python version Upload date Hashes
Filename, size multiprocessing_logging-0.3.0-py2.py3-none-any.whl (7.9 kB) File type Wheel Python version py2.py3 Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page