out 0.60

Fun take on logging for non-huge projects. Gets "outta" the way.

Fun take on logging for non-huge projects—gets “outta” the way.

(Why’s are covered in the background section at the bottom.)

Install

⏵ pip3 install out  # or out[highlight]

Features

First of all, out is concise as hell, basically a singleton logger ready on import. In interactive mode:

>>> import out

>>> out('And away we go…')  # configurable default level
🅸 main/func:1 And away we go…

>>> out.warn('Danger Will Robinson!')
🆆 main/func:1 Danger Will Robinson!

(Imagine with nice ANSI colors. 😁) Out has simple themes for message formats, styles, and icons. Not to worry, out is more conservative in production mode, turned on automatically by redirecting stderr:

⏵ python3 script.py |& cat  # bash, fish: ^|
2018-09-10 17:18:19.123 ✗ ERROR main/func:1 Kerblooey!

Colors, Highlighting, Unicode Icons

• Colors are ready to go in interactive mode, and turn off automatically when output is redirected.

• Unicode symbols are used throughout as “icons” for increased readability and conciseness.

• Syntax highlighting of data structures (oft parsed from remote APIs) is available too, via Pygments.

Useful defaults, and easy to configure!

>>> out.configure(
        level='note',           # or int
        default_level='info',   # out('…')
        datefmt='…',            # strftime
        msgfmt='…',             # see below
        stream=file,            # stderr

        theme=name|dict,        # see below
        icons=name|dict,        #   about themes
        style=name|dict,
        lexer='python3',        # highlighting
    )

Note

This is a library to simplify logging for the main script of applications.

Libraries/modules should continue on as they always have:

import logging

log = logging.getLogger(__name__)
# do not configure loggers, just use:
log.debug('foo')

Levels++

While the standard levels continue to exist (NOTSET, DEBUG, INFO, WARNING, ERROR, CRITICAL). A few additions and slight modifications have been made. Commonly requested:

• TRACE, for absurdly voluminous data, perhaps system calls or network traffic.

• NOTE, for positive messages that should/must be shown by default—unlike the standard warning, which could encourage the viewer to worry. e.g.:

  NOTE - Token is ABCXYZ, rather than…

  WARNING - Token is ABCXYZ.

• EXCEPT, to differentiate common from unexpected errors. Think FileNotFound vs. Exception.

• FATAL, a renaming of CRITICAL, since that name is long, pushes alignment, and does not capture message intent as well as fatal. Std-lib already allows this but still labels it critical on output. Out does not.

Templating

Log Format:

By default out supports the {} formatting style for both the log template and message fields, as it is a bit easier to read. Most fields are found in the Python logging docs.:

{asctime}           Textual time when the LogRecord created.
{msecs}             Millisecond portion of the creation time
{filename}          Filename portion of pathname
{funcName}          Function name
{lineno)            Source line number where called.
{levelno}           Numeric logging level for the message
{levelname}         Text logging level for the message
{pathname}          Full path of the source file called.
{message}           The result of record.getMessage().
{module}            Module (name portion of filename)
{name}              Name of the logger (logging channel)

Use of out.format.ColorFormatter adds these additional fields:

{on}{icon}{off}     Level-style and icon support.

For example:

out.configure(
    msgfmt='{on}{icon}{levelname:<7}{off} {message}'
)

Message:

When writing messages, printf % formatting style is supported as well due to compatibility requirements with a majority of libraries:

out.warn('foo: %s', bar)
out.warn('foo: {}', bar)

The second form may be used also, though it will be a tiny bit slower, since the printf style is tried first.

DateTime Format

These are configuable via strftime syntax and the datefmt keyword to configure.

Themes

Themes are simply dictionaries with one entry per level:

>>> from out.themes import themes, icons, styles

>>> icons['circled']  # Unicode
{'TRACE': '🅣', 'DEBUG': '🅓', 'INFO': '🅘', 'WARNING': '🅦',
 'NOTE': '🅝', 'ERROR': '🅔', 'EXCEPT': '🅧', 'CRITICAL': '🅕',
 'FATAL': '🅕', 'NOTSET': '🅝'}

>>> styles['blink']  # ANSI escapes
{'TRACE': '\x1b[35m', 'DEBUG': '\x1b[34m', 'INFO': '\x1b[32m',
 'WARNING': '\x1b[93m', 'NOTE': '\x1b[96m', 'ERROR': '\x1b[31m',
 'EXCEPT': '\x1b[91m', 'CRITICAL': '\x1b[97m',
 'FATAL': '\x1b[97;5m', 'NOTSET': '\x1b[0m'}

The console package is a good choice to generate ANSI styles for the levels, as well as styling other fields:

from console import fg, bg, fx
import out

blue_note = dict(
    NOTE=str(fg.lightblue + fx.bold + fx.reverse),
    # other levels…
)
out.configure(
    style=blue_note,  # ← level styles, field styles ↓
    msgfmt=bg.blue('{asctime}') + ' {message}',
)
out.note('John Coltrane')

A full theme is the whole kit together in a mapping—styles, icons, and templates:

>>> interactive = {
 'style': {},  # level:value mapping, see above
 'icons': {},  # level:value
 'fmt': '{asctime} {icon} {message}',  # message format
 'datefmt': '%H:%M:%S',  # date format,
}

Using Themes

In the configure method of the out logger, to use a theme from the themes module, simply specify one by name:

>>> out.configure(
        theme='production',
    )

Or by setting a custom mapping:

>>> out.configure(
        theme=interactive,  # or perhaps just icons:
        icons=dict(DEBUG='• ', INFO='✓ ', WARNING='⚠ ', ) # …
    )

A few themes are bundled:

Icons:

ascii, ascii_symbol, circled, circled_lower, rounded, symbol

Styles:

• norm

• bold

• mono (monochrome)

• blink (fatal error only)

Full themes:

• interactive

• production

• plain (Uses logging.Formatter for lower overhead.)

• json (Uses formatter.JSONFormatter)

• mono (monochrome)

• linux_interactive, linux_production (vga console)

Note

When there are conflicting arguments to the configure method, the last specified will win. This requires > Python 3.6, due to ordered keyword args. Below this version it is not recommended to do this since keyword order will be undefined and therefore the result. One workaround is to call configure twice.

Syntax Highlighting w/Pygments

When Pygments is installed, syntax highlighting is available for Python data structures and code, as well as JSON and XML strings—potentially anything Pygments can highlight. This can be helpful when debugging remote APIs for example.

A lexer may be selected by name via configure(lexer=LEXER_NAME), disabled by setting to None.

Use:

Message text following a {, [, <, or ' char is highlighted with the current lexer+formatter:

# default Python3
out.debug('PYON data: %s',
          {'data': [None, True, False, 123]})

out.configure(lexer='json')
out.debug('JSON data: '
          '{"data": [null, true, false, 123]}')

(Imagine with lovely ANSI flavors. 😁)

Tips

• By default the logger prints to stderr. The reason being that when used in an interactive script normal application output may be easily segregated from log messages during redirection.

  Configurable via the stream keyword to .configure().

• Upgrading a long script from print() is easy:

  import out
  
  print = out.info  # or other level

  Or perhaps some logging was already added, but you’d like to downsize. Add this to your main script:

  import out as logger

  Less code will need to be changed.

• The ColorFormatter and JSONFormatter classes can be used in your own project:

  >>> from out.format import ColorFormatter
  
  >>> cf = ColorFormatter()
  >>> handler.setFormatter(cf)

• To print the current logging configuration:

  >>> out.log_config()  # quotes to shut off highlighting:
  '''
  🅳  Logging config:
  🅳  / name: main, id: 139973461370360
  🅳    .level: trace (7)
  🅳    .default_level: info (20)
  🅳    + Handler: 0 <StreamHandler <stderr> (NOTSET)>
  🅳      + Formatter: <out.format.ColorFormatter object at 0x7f4e1c65efd0>
  🅳        .style: <logging.StrFormatStyle object at 0x7f4e1c65ef28>
  🅳        .datefmt: '%H:%M:%S'
  🅳        .msgfmt: '  {on}{icon}{off} {message}'
  '''

The logger in the main script file is named “main,” also known as the “root” logger.

Background

If you’re here it’s very likely you already know that the Python standard logging module is extremely flexible, and that’s great and all. Unfortunately, it is overkill for small to medium projects, and these days many larger ones too. Additionally, its various Java-isms grate on the nerves, accentuating a big enterprisey design.

Meanwhile, the rise of 12 Factor App patterns for daemons and services means that simply logging to stdout/err is expected and desired for portability:

A twelve-factor app never concerns itself with routing or storage of its output stream. It should not attempt to write to or manage logfiles. Instead, each running process writes its event stream, unbuffered, to stdout. During local development, the developer will view this stream in the foreground of their terminal to observe the app’s behavior.

Therefore, for many (if not most) applications, all the complexity and mumbo-jumbo in the logging package documentation about multiple loggers with different levels, different handlers, formatters, adapters, filters, rotation, and complex configuration is flexibility at the wrong level. In fairness, this may not have always been the case, and can still be helpful, perhaps on Windows.

Additionally, logging tools have also become standardized over time, handling cross-language and cross-platform messages. Imagine a pipeline where log events are routed and multiple tools can be plugged in or out as needed—organization-wide rather than app- or language-wide.

So, unless you have unique requirements, there’s no need to reimplement logrotate, syslog, systemd, and proprietary metrics tools in every programming language. Just blast those logs to stdout/stderr and get logging outta the way!

Enter the out project. It’s ready to start logging from the get go. It uses Python’s standard logging infrastructure by default, so is still quite flexible when need be.

Well, you’ve heard this before. However, out tries a bit harder create a fun, easy-to-use interface, as discussed above.

Naming

Regarding the name, well of course would have liked something along the lines of log but all variations of that are long gone on PyPI. out() is a name I’ve often used over the years as a poor-man’s logger—really a functional wrapper around print, until I could get around to adding proper logging. Now, the tradition continues. The name is short, simple, and conceptually fits, if a little bland.