How to log in Python applications

If you already know how to log and just want to use the LoggingInitializer, see :ref: using-LI.

At the start of each module, import the Logging library and generate a logger:

import logging
logger = logging.getLogger(__name__)

This generates an object which knows from which module you’re logging. It will reference the module in the log.

In your code, tell the logger to generate log messages where needed:

logger.debug("Going to do something important now.")
try:
  something_important()
except:
  logger.error("Something important didn't work!!!")
logger.info("Something important is done.")

The methods debug(), info() and error() (we also have warning() and critical()) generate log messages at different log levels. Log levels are used to communicate the importance of a log line, to keep logs relevant, and as small as possible (but not smaller).

You will probably choose different log levels on different environments: on a production server, most teams will only log “WARNING”, “ERROR” and “CRITICAL”; on a testing server usually also “INFO” so it’s easier to follow what’s going on, and when debugging… “DEBUG”. When writing log messages, choose your log levels with this in mind.

The default setting for LoggingInitializer is INFO, which means that statements logged with logger.debug() will not be logged by default.

For more information on how to use the Python logger and info on the different log levels, see this helpful tutorial in the Python docs.

Using LoggingInitializer

Include the LoggingInitializer in your application to quickly set parameters for the Python logger.

At the start of your program, populate a LoggingConfig object with the required parameters.

Below an example of a logger that writes to StdOut and to a file is portrayed. The user executing the program should have write rights on the specified directory!

config = {
          "identifier": "my_lovely_program",
          "directory": "/var/logs",
      }

logging_config = LoggingConfig(**config)

Then create the LoggingInitializer object:

logging_initializer = LoggingInitializer(logging_config)

If you want to suppress the logging to StdOut (only log to a file), use Quiet Mode:

logging_initializer = LoggingInitializer(logging_config, quiet=True)

If you want to suppress the logging to a file (only log to StdOut), simply don’t specify a directory in the LoggingConfig.

logging_config = LoggingConfig(identifier="your-package-name")
logging_initializer = LoggingInitializer(logging_config)

For more info on setting the log format and date format, see the Python docs. If another application is ingesting the logs, do not change the format unannounced.

Installation

To install logging-initializer, do:

git clone https://gitlab.com/rwsdatalab/projects/logging-initializer.git
cd logging-initializer
pip install .

Run tests (including coverage) with:

pip install -r requirements-dev.txt
python setup.py test

Documentation

Find the full documentation here.

License

Copyright (c) 2022-2023, Rijkswaterstaat