uvlog 0.1.3

Pythonic logger with better performance and contextvars + JSON support out of the box

uvlog is yet another logging library built with an idea of a simple logger what 'just works' without need for extension and customization.

• Single package, no other dependencies
• JSON and contextvars out of the box
• Less abstraction, better performance
• Pythonic method names and classes

Use

Our main scenario is logging our containerized server applications, i.e. writing all the logs to the stderr of the container, where they are gathered and sent to the log storage by another service. However, you can use this library for any application as long as it does not require very complicated things like complex filters, adapters etc.

The easiest way to access a logger is similar to the standard module. Note that you can pass extra variables as keyword arguments while logging.

from uvlog import get_logger

logger = get_logger('app')
logger.set_level('DEBUG')

logger.info('Hello, {name} {surname}!', name='John', surname='Dowe')

To write an exception use exc_info param.

try:
    ...
except ValueError as exc:
    logger.error('Something bad happened', exc_info=exc)

Configuration is possible in a way similar to dictConfig. The configuration dict itself is JSON compatible. The configure() function returns the root logger instance. Note that one destination can be assigned to only one handler, but each logger can have any number of handlers.

Here's an extensive example of such config.

from uvlog import configure

logger = configure({
    'loggers': {
        '': {  # the root logger
            'level': 'DEBUG',
            'handlers': ['stderr', '/etc/log.txt']
        }
    },
    'handlers': {
        'stderr': {  # 'stderr' and 'stdout' are reserved for these special destinations
            'class': 'StreamHandler',
            'formatter': 'json'
        },
        '/etc/log.txt': {
            'class': 'QueueStreamHandler',
            'formatter': 'text'
        }
    },
    'formatters': {
        'text': {
            'class': 'TextFormatter',
            'format': '{asctime} : {name} : {message}',  # see `LogRecord` for the list of fields
            'timestamp_separator': ' '                   # by default it's 'T'
        },
        'json': {
            'class': 'JSONFormatter',
            'keys': ['asctime', 'name', 'message']       # see `LogRecord` for the list of fields
        }
    }
})

app_logger = logger.get_child('app', persistent=True)

You can use context variables to maintain log context between log records. This can be useful for log aggregation. See the documentation on contextvars for more info.

from uvlog import LOG_CONTEXT, get_logger

app_logger = get_logger('app')

async def handler_request(request):
    LOG_CONTEXT.set({'request_id': request.headers['Request-Id']})
    await call_system_api()

async def call_system_api():
    # this record will have 'request_id' in its context
    app_logger.info('Making a system call')

When using the JSONFormatter you should consider providing a better json serializer for better performance (such as orjson).

import orjson
from uvlog import JSONFormatter

JSONFormatter.serializer = orjson.dumps

Never say never

The library adds support for additional log level - NEVER. The idea behind this is to use such logs in places of code which should never be executed in production and monitor such cases. 'NEVER' logs have the maximum priority. They cannot be suppressed by any logger and are always handled.

The use of NEVER is straightforward.

def handle_authorization(username, password) -> bool:
    if DEBUG and username == debug_login:
        logger.never('skip authorization for {username}', username=username)
        return True
    return check_password_is_valid(username, password)

Why not just use a DEBUG or WARNING level here? The reason is low priority of such records, which allows them to be mixed with less significant logs or even be skipped by loggers.

Loggers are weak

Unlike the standard logging module, loggers are weak referenced unless they are described explicitly in the configuration dict or created with persistent=True argument.

It means that a logger is eventually garbage collected once it has no active references. This allows creation of a logger per task, not being concerned about running out of memory eventually. However, this also means that all logger settings for a weak logger will be forgotten once it's collected.

In general this is not a problem since you shouldn't fiddle with logger settings outside the initialization phase.

Sampling

The library implements internal log sampling. In shorts, it allows you to specify the sample_rate, a probability at which a logger will pass a record to the handlers. It allows to release some load due to extensive logging.

from uvlog import get_logger

logger = get_logger()
logger.sample_rate = 0.25

... or via a config dict

from uvlog import configure

configure({
    'loggers': {
        '': {
            'sample_rate': 0.25
        }
    }
})

See the documentation on sampling for more info.

Customization

You can create custom formatters and handlers with ease. Note that inheritance is not required. Just be sure to implement Handler / Formatter protocol.

See the extension guide for more info. There's an example of HTTP queue logger using requests library there.

Performance

Benchmark results are provided for the M1 Pro Mac (16GB). The results are for the StreamHandler writing same log records into a file. The QueueStreamHandler provides similar performance, but has been excluded from the test since Python threading model prevents results from being consistent between runs. However, I'd still recommend using the QueueStreamHandler for server applications.

name	total (s)	logs/s	%
python logger	0.085	117357	100
uvlog text	0.022	455333	388
uvlog json	0.015	665942	567

Compatibility

There's a certain compatibility between this logger and the standard logger. However, it's impossible to preserve full compatibility because of certain design decisions.

See the compatibility guide if you want to migrate from the standard python logger to this one.

Ideas / goals

• Rotating file handlers
• Asynchronous queue logging to HTTP / TCP / UDP
• Better customization for Logger / LogRecord objects
• Better coverage
• Cython?