Skip to main content

A Python package which supports global logfmt formatted logging.

Project description

Python Logfmter

pre-commit test python-3.6-3.7-3.8-3.9-3.10

A Python package which supports global logfmt formatted logging.

  1. Install
  2. Usage
    1. Integration
    2. Configuration
    3. Extend
  3. Development
    1. Required Software
    2. Getting Started
    3. Publishing

Install

$ pip install logfmter

Usage

Before integrating this library, you should be familiar with Python's logging functionality. I recommend reading the Basic Logging Tutorial.

This package exposes a single Logfmter class that can be integrated into the standard library logging system similar to any logging.Formatter.

The provided formatter will logfmt encode all logs. Key value pairs are provided via the extra keyword argument or by passing a dictionary as the log message. To prevent invalid keys from breaking the logfmt style, the formatter normalizes all keys:

  • replaces empty keys with an underscore
  • replaces spaces with underscores
  • escapes newlines

If a log message is created via logging.exception (inside an exception handler), then the exception information (traceback, type, and message) will be encoded in the exc_info parameter.

Integration

basicConfig

import logging
from logfmter import Logfmter

handler = logging.StreamHandler()
handler.setFormatter(Logfmter())

logging.basicConfig(handlers=[handler])

logging.error("hello", extra={"alpha": 1}) # at=ERROR msg=hello alpha=1
logging.error({"token": "Hello, World!"}) # at=ERROR token="Hello, World!"

dictConfig

import logging.config

logging.config.dictConfig(
    {
        "version": 1,
        "formatters": {
            "logfmt": {
                "()": "logfmter.Logfmter",
            }
        },
        "handlers": {
            "console": {"class": "logging.StreamHandler", "formatter": "logfmt"}
        },
        "loggers": {"": {"handlers": ["console"], "level": "INFO"}},
    }
)

logging.info("hello", extra={"alpha": 1}) # at=INFO msg=hello alpha=1

Notice, you can configure the Logfmter by providing keyword arguments as dictionary items after "()":

...

    "logfmt": {
        "()": "logfmter.Logfmter",
        "keys": [...],
        "mapping": {...}
    }

...

Configuration

keys

By default, the at=<levelname> key/value will be included in all log messages. These default keys can be overridden using the keys parameter. If the key you want to include in your output is represented by a different attribute on the log record, then you can use the mapping parameter to provide that key/attribute mapping.

Reference the Python logging.LogRecord Documentation for a list of available attributes.

import logging
from logfmter import Logfmter

formatter = Logfmter(keys=["at", "processName"])

handler = logging.StreamHandler()
handler.setFormatter(formatter)

logging.basicConfig(handlers=[handler])

logging.error("hello") # at=ERROR processName=MainProceess msg=hello

mapping

By default, a mapping of {"at": "levelname"} is used to allow the at key to reference the log record's levelname attribute. You can override this parameter to provide your own mappings.

import logging
from logfmter import Logfmter

formatter = Logfmter(
    keys=["at", "process"],
    mapping={"at": "levelname", "process": "processName"}
)

handler = logging.StreamHandler()
handler.setFormatter(formatter)

logging.basicConfig(handlers=[handler])

logging.error("hello") # at=ERROR process=MainProceess msg=hello

datefmt

If you request the asctime attribute (directly or through a mapping), then the date format can be overridden through the datefmt parameter.

import logging
from logfmter import Logfmter

formatter = Logfmter(
    keys=["at", "when"],
    mapping={"at": "levelname", "when": "asctime"},
    datefmt="%Y-%m-%d"
)

handler = logging.StreamHandler()
handler.setFormatter(formatter)

logging.basicConfig(handlers=[handler])

logging.error("hello") # at=ERROR when=2022-04-20 msg=hello

Extend

You can subclass the formatter to change its behavior.

import logging
from logfmter import Logfmter


class CustomLogfmter(Logfmter):
    """
    Provide a custom logfmt formatter which formats
    booleans as "yes" or "no" strings.
    """

    @classmethod
    def format_value(cls, value):
        if isinstance(value, bool):
            return "yes" if value else "no"

	return super().format_value(value)

handler = logging.StreamHandler()
handler.setFormatter(CustomLogfmter())

logging.basicConfig(handlers=[handler])

logging.error({"example": True}) # at=ERROR example=yes

Development

Required Software

Refer to the links provided below to install these development dependencies:

Getting Started

Setup

$ <runtimes.txt xargs -n 1 pyenv install -s
$ direnv allow
$ pip install -r requirements/dev.txt
$ pre-commit install
$ pip install -e .

Tests

Run the test suite against the active python environment.

$ pytest

Run the test suite against the active python environment and watch the codebase for any changes.

$ ptw

Run the test suite against all supported python versions.

$ tox

Publishing

Create

  1. Update the version number in logfmter/__init__.py.

  2. Add an entry in HISTORY.md.

  3. Commit the changes, tag the commit, and push the tags:

    $ git commit -am "v<major>.<minor>.<patch>"
    $ git tag v<major>.<minor>.<patch>
    $ git push origin main --tags
    
  4. Convert the tag to a release in GitHub with the history entry as the description.

Build

$ python -m build

Upload

$ twine upload dist/*

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

logfmter-0.0.6.tar.gz (6.3 kB view hashes)

Uploaded source

Built Distribution

logfmter-0.0.6-py3-none-any.whl (7.0 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page