Skip to main content

Logger for multiprocessing applications

Project description


Supported Python versions License

When using the multiprocessing module, logging becomes less useful since sub-processes should log to individual files/streams or there's the risk of records becoming garbled.

This simple module implements a Handler that when set on the root Logger will handle tunneling the records to the main process so that they are handled correctly.

It's currently tested in Linux and Python 2.7 & 3.6+.

Pypy3 hangs on the tests so I don't recommend using it.

Pypy appears to be working, recently.

Only works on POSIX systems and only Linux is supported. It does not work on Windows.


This library was taken verbatim from a StackOverflow post and extracted into a module so that I wouldn't have to copy the code in every project.

Later, several improvements have been contributed.


Before you start logging but after you configure the logging framework (maybe with logging.basicConfig(...)), do the following:

import multiprocessing_logging


and that's it.

With multiprocessing.Pool

When using a Pool, make sure install_mp_handler is called before the Pool is instantiated, for example:

import logging
from multiprocessing import Pool
from multiprocessing_logging import install_mp_handler

pool = Pool(...)


The approach of this module relies on fork being used to create new processes. This start method is basically unsafe when also using threads, as this module does.

The consequence is that there's a low probability of the application hanging when creating new processes.

As a palliative, don't continuously create new processes. Instead, create a Pool once and reuse it.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

multiprocessing_logging-0.3.4-py2.py3-none-any.whl (8.8 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page