A Python package which supports global logfmt formatted logging.
Project description
Python Logfmter
A Python package which supports global logfmt formatted logging.
Install
$ pip install logfmter
Usage
Before integrating this library, you should be familiar with Python's logging functionality. I recommend reading the Basic Logging Tutorial.
This package exposes a single Logfmter
class that can be integrated into
the standard library logging system similar to any logging.Formatter
.
The provided formatter will logfmt encode all logs. Key value pairs are provided
via the extra
keyword argument or by passing a dictionary as the log message.
To prevent invalid keys from breaking the logfmt style, the formatter normalizes all keys:
- replaces empty keys with an underscore
- replaces spaces with underscores
- escapes newlines
If a log message is created via logging.exception
(inside an exception handler), then
the exception information (traceback, type, and message) will be encoded in the
exc_info
parameter.
Integration
import logging
from logfmter import Logfmter
handler = logging.StreamHandler()
handler.setFormatter(Logfmter())
logging.basicConfig(handlers=[handler])
logging.error("hello", extra={"alpha": 1}) # at=ERROR msg=hello alpha=1
logging.error({"token": "Hello, World!"}) # at=ERROR token="Hello, World!"
import logging.config
logging.config.dictConfig(
{
"version": 1,
"formatters": {
"logfmt": {
"()": "logfmter.Logfmter",
}
},
"handlers": {
"console": {"class": "logging.StreamHandler", "formatter": "logfmt"}
},
"loggers": {"": {"handlers": ["console"], "level": "INFO"}},
}
)
logging.info("hello", extra={"alpha": 1}) # at=INFO msg=hello alpha=1
Notice, you can configure the Logfmter
by providing keyword arguments as dictionary
items after "()"
:
...
"logfmt": {
"()": "logfmter.Logfmter",
"keys": [...],
"mapping": {...}
}
...
Configuration
keys
By default, the at=<levelname>
key/value will be included in all log messages. These
default keys can be overridden using the keys
parameter. If the key you want to include
in your output is represented by a different attribute on the log record, then you can
use the mapping
parameter to provide that key/attribute mapping.
Reference the Python logging.LogRecord
Documentation
for a list of available attributes.
import logging
from logfmter import Logfmter
formatter = Logfmter(keys=["at", "processName"])
handler = logging.StreamHandler()
handler.setFormatter(formatter)
logging.basicConfig(handlers=[handler])
logging.error("hello") # at=ERROR processName=MainProceess msg=hello
mapping
By default, a mapping of {"at": "levelname"}
is used to allow the at
key to reference
the log record's levelname
attribute. You can override this parameter to provide your
own mappings.
import logging
from logfmter import Logfmter
formatter = Logfmter(
keys=["at", "process"],
mapping={"at": "levelname", "process": "processName"}
)
handler = logging.StreamHandler()
handler.setFormatter(formatter)
logging.basicConfig(handlers=[handler])
logging.error("hello") # at=ERROR process=MainProceess msg=hello
datefmt
If you request the asctime
attribute (directly or through a mapping), then the date format
can be overridden through the datefmt
parameter.
import logging
from logfmter import Logfmter
formatter = Logfmter(
keys=["at", "when"],
mapping={"at": "levelname", "when": "asctime"},
datefmt="%Y-%m-%d"
)
handler = logging.StreamHandler()
handler.setFormatter(formatter)
logging.basicConfig(handlers=[handler])
logging.error("hello") # at=ERROR when=2022-04-20 msg=hello
Extend
You can subclass the formatter to change its behavior.
import logging
from logfmter import Logfmter
class CustomLogfmter(Logfmter):
"""
Provide a custom logfmt formatter which formats
booleans as "yes" or "no" strings.
"""
@classmethod
def format_value(cls, value):
if isinstance(value, bool):
return "yes" if value else "no"
return super().format_value(value)
handler = logging.StreamHandler()
handler.setFormatter(CustomLogfmter())
logging.basicConfig(handlers=[handler])
logging.error({"example": True}) # at=ERROR example=yes
Development
Required Software
Refer to the links provided below to install these development dependencies:
Getting Started
Setup
$ <runtimes.txt xargs -n 1 pyenv install -s
$ direnv allow
$ pip install -r requirements/dev.txt
$ pre-commit install
$ pip install -e .
Tests
Run the test suite against the active python environment.
$ pytest
Run the test suite against the active python environment and watch the codebase for any changes.
$ ptw
Run the test suite against all supported python versions.
$ tox
Publishing
Create
-
Update the version number in
logfmter/__init__.py
. -
Add an entry in
HISTORY.md
. -
Commit the changes, tag the commit, and push the tags:
$ git commit -am "v<major>.<minor>.<patch>" $ git tag v<major>.<minor>.<patch> $ git push origin main --tags
-
Convert the tag to a release in GitHub with the history entry as the description.
Build
$ python -m build
Upload
$ twine upload dist/*
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.