uvlog 0.1.0

Pythonic logger with better performance and contextvars + JSON support out of the box

[ [ [ [ [

uvlog is yet another logging library built with an idea of a simple logger what 'just works' without need for extension and customization.

• Single file, no external dependencies
• JSON and contextvars out of the box
• Less abstraction, better performance
• Pythonic method names and classes

Use cases

Our main scenario is logging our containerized server applications, i.e. writing all the logs to the stderr of the container, where they are gathered and sent to the log storage by another service. However, you can use this library for any application as long as it does not require very complicated things like complex filters, adapters etc.

Usage

The easiest way to access a logger is similar to the standard module. Note that you can pass extra variables as keyword arguments while logging.

from uvlog import get_logger

logger = get_logger('app')
logger.set_level('DEBUG')
logger.debug('Hello, world!', name='John', surname='Dowe')

To write an exception use exc_info param.

try:
    ...
except ValueError as exc:
    logger.error('Something bad happened', exc_info=exc)

Configuration is possible in a way similar to dictConfig. The configuration dict itself is JSON compatible. The configure() function returns the root logger instance. Note that one destination can be assigned to only one handler, but each logger can have any number of handlers.

from uvlog import configure

logger = configure({
    'loggers': {
        '': {
            'levelname': 'DEBUG',
            'handlers': ['stderr', '/etc/log.txt']
        }
    },
    'handlers': {
        'stderr': {
            'class': 'StreamHandler',
            'formatter': 'json'
        },
        '/etc/log.txt': {
            'class': 'QueueHandler',
            'formatter': 'text'
        }
    },
    'formatters': {
        'text': {
            'class': 'TextFormatter',
            'format_string': '{asctime} : {name} : {message}',
            'timestamp_separator': ' '
        },
        'json': {
            'class': 'JSONFormatter',
            'keys': ['asctime', 'name', 'message']
        }
    }
})

app_logger = logger.get_child('app', persistent=True)

You can use context variables to maintain log context between log records. This can be useful in the server environment where you can assign a unique Request-Id to each request and easily aggregate the logs for it afterward.

from uvlog import LOG_CONTEXT, get_logger

app_logger = get_logger('app')

async def handler_request(request):
    LOG_CONTEXT.set({'request_id': request.headers['Request-Id']})
    await call_system_api()

async def call_system_api():
    # this log record will have 'request_id' in its context
    app_logger.info('Making a system call')

When using the JSONFormatter you should consider providing a better json serializer for better performance (such as orjson).

import orjson
from uvlog import JSONFormatter

JSONFormatter.serializer = orjson.dumps

Loggers are weak

Unlike the standard logging module, loggers are weak referenced unless they are described explicitly in the configuration dict or created with persistent=True argument.

It means that a logger is eventually garbage collected once it has no active references. This allows creation of a logger per task, not being concerned about running out of memory eventually. However, this also means that all logger settings for a weak logger will be forgotten once it's collected.

In general this is not a problem since you shouldn't fiddle with logger settings outside the initialization phase.

Customization

You can create custom formatters and handlers with ease. Note that inheritance is not required. Just be sure to implement Handler / Formatter protocol.

from uvlog import add_formatter_type, add_handler_type

class IdioticFormatter:
    
    def format(self, record, /) -> bytes:
        return b'Bang!'

add_formatter_type(IdioticFormatter)

Custom logger class can be set using set_logger_type

from uvlog import set_logger_type, Logger

class MyLogger(Logger):
    ...

set_logger_type(MyLogger)

Performance

Benchmark results are provided for the M1 Pro Mac (16GB). The results are for the StreamHandler writing same log records into a file. The QueueHandler provides similar performance, but has been excluded from the test since Python threading model prevents results from being consistent between runs. However, I'd still recommend using the QueueHandler for server applications.

name	total (s)	logs/s	%
python logger	0.11	92568	100
uvlog text	0.031	325034	351
uvlog json	0.02	500878	541

Ideas / goals

• Rotating file handlers
• Asynchronous queue logging to HTTP / TCP / UDP
• Referencing extra and ctx keys in log message format strings
• Better customization for Logger / LogRecord objects
• Cython?