Helpful code for logging in Python
Project description
powerflex-logging-utilities
Helpful code for logging in Python by PowerFlex.
Module | Description |
---|---|
forbid_toplevel_logging | Disable logging with the top-level root logging functions such as logging.info . |
log_slow_callbacks | Either warn or info log when an async callback runs for too long. |
init_loggers | A function for easily setting up logging to a file and to stdout. |
Class | Description |
---|---|
JsonFormatter | A JSON log formatter to enable structured logging. It depends on the python-json-logger package. |
TraceLogger | A Python Logger subclass that adds a TRACE logging level |
AsyncNatsLogLevelListener | A NATS interface for changing the program's log level by sending a NATS request |
Sample usage
Initializing Loggers
Setup all Loggers to output JSON to stdout and to a file:
import logging
import sys
from powerflex_logging_utilities import (
JsonFormatter,
TraceLogger,
init_loggers,
)
LOG_LEVEL = "DEBUG"
FILE_LOG_LEVEL = "TRACE"
LOG_FILE = "./logs/trace.log"
MAX_LOG_FILE_MB = 10
MAX_TOTAL_LOG_FILE_MB = 10000
root_logger = logging.getLogger()
# Log warnings with the py.warnings logger
logging.captureWarnings(True)
# Fix iPython autocomplete
logging.getLogger("parso").propagate=False
init_loggers.init_loggers(
[root_logger],
log_level=LOG_LEVEL,
file_log_level=FILE_LOG_LEVEL,
filename=LOG_FILE,
max_bytes=1000 * 1000 * MAX_LOG_FILE_MB,
backup_count=MAX_TOTAL_LOG_FILE_MB // MAX_LOG_FILE_MB,
stream=sys.stdout,
formatter=JsonFormatter,
info_logger=root_logger,
)
logger = logging.getLogger(__name__)
This uses Python's logger propagation feature. We only need to configure the root Logger in order to make sure all other Loggers output in the desired format.
You can pass formatter_kwargs
to enable logging with a different JSON serializer.
To use:
logger = logging.getLogger(__name__)
logger.info("hello world")
Explicitly listing loggers
You can also list the loggers you'd like to configure instead of configuring the root logger.
This could be useful if you configure your package's main logger
logging.getLogger("package")
. You can then use Python's logger propagation by calling
logging.getLogger("package.submodule.a.b.c")
to get Logger instances for all
other submodules.
import logging
from powerflex_logging_utilities import (
JsonFormatter,
TraceLogger,
init_loggers,
)
logger = logging.getLogger("your_package_name")
# Log warnings with the py.warnings logger
logging.captureWarnings(True)
init_loggers.init_loggers(
[logger, "asyncio", "py.warnings"],
log_level="DEBUG",
file_log_level="TRACE",
filename="./logs/trace-no-root.log",
formatter=JsonFormatter,
info_logger=logger,
)
NOTICE: if you use this method, any loggers you do not explicitly list will have non-JSON output.
Using several other utilities
import logging
from powerflex_logging_utilities import (
forbid_toplevel_logging,
log_slow_callbacks,
)
logger = logging.getLogger(__name__)
# Log slow async callbacks with two log levels
log_slow_callbacks.log_slow_callbacks(logger)
# Forbid functions such as logging.info since they implicitly use the root logger
forbid_toplevel_logging.forbid_logging_with_logging_toplevel()
Using the JSON formatter
import logging
import sys
from powerflex_logging_utilities import JsonFormatter
log_handler = logging.StreamHandler(stream=sys.stdout)
log_handler.setLevel("DEBUG")
log_handler.setFormatter(JsonFormatter())
logger = logging.getLogger(__name__)
logger.addHandler(log_handler)
logger.setLevel("DEBUG")
logger.info("hello world", extra={
"data": ["log structured data", ":D"],
1: "handles non string key",
})
{
"message": "hello world",
"name": "__main__",
"module": "<ipython-input-10-b016ce80d46f>",
"lineno": 1,
"funcName": "<cell line: 1>",
"filename": "<ipython-input-10-b016ce80d46f>",
"asctime": "2022-05-12 01:04:16,824",
"data": [
"log structured data",
":D"
],
"severity": "INFO",
"1": "handles non string key"
}
Using pipenv
- Run
make setup-with-pipenv
to install all dependencies. Make sure you have the version of Python specified in.tool-versions
or simply change this file to your Python version (must be 3.8+). - Run
pipenv shell
or run the followingmake
commands withpipenv run make ...
. You could also aliaspmake
topipenv run make
for convenience.
Tests
There is 100% code coverage.
make test-unit
To test in several versions of Python, run:
tox
Testing the code in this README
make test-readme
Checking code quality
The Github Actions will run all of the following checks on the code.
Code formatting
make format-fix
Linting
make lint
Type checking
make type-check-strict
Releasing to pypi
- Make sure you have committed all code you wish to release.
- Make sure all code checks have passed.
- Set the version in
./src/powerflex_logging_utilities/VERSION
Please attempt to follow semantic versioning. - Run
make bump-version
- Run
make release
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for powerflex-logging-utilities-1.2.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8cd86c2d7c423f34edeaf0c98e87d804ac2495058c1dea3ea78c8495c74de1a |
|
MD5 | 3a9c1c4a046bcda6285bb52f7d50b9e7 |
|
BLAKE2b-256 | c64e76230872cc81bd1e2d302d71c9f770832e583f7f7b27ef23a9daee3f8f1c |
Hashes for powerflex_logging_utilities-1.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e13efea1679ee89094e3ad5b5d20c735abe6b62f2209d52dac6e249285c0033f |
|
MD5 | b87ef128e6e3eed355cc8dc9ea3c998b |
|
BLAKE2b-256 | b1766b4da18478dddbfe3936bed7e8127e8a24176a63a499914ac4e3d531bf47 |