Skip to main content

Nested telemetry logger.

Project description

Austen

Nested telemetry logger. Perfect for printing algorithms. Dumps into json file.

Features

Core idea behind Austen was to have algorithm call stack logged with exact arguments.These are stored in a dictionary and dumped into json. Additionaly Austen handles dumps of some additional file types. These include:

  • plots
  • sheets
  • images
  • objects (using joblib)

All files will be dumped in a folder structure, that corresponds to your algorithm (folder per branch).

Getting started

pip install austen

How to use

Manual entries

We'll start by importing Logger class from the library.

from austen import Logger

Now we can instantiate Logger object.

logger = Logger(output='logs')

By output argument you can specify base directory for subsequent dumps. It will dump to your current working directory (relative path). If you want to dump it somewhere else, just put absolute path in there. Now it'll dump to ./logs/.

Now let's add some entry to our logger.

logger.add_entry('foo', 'bar')

You can also add multiple entries at once.

entries = [
    ('a', 1),
    ('b', 2),
    ('c', 5)
]

logger.add_entries(entries)

Entries are stored in a dictionary. This means that key has to be hashable. If you think something is appropriate for json format, just push it to logger.

Wrapping function calls

Let's log some functions.

def add(x, y):
    return x + y

def foo(logger:Logger, x, y):
    return logger.log_func(
        func=add,
        kwargs={'x': x, 'y': y}
    )

Other file types

Now let's see how we can dump files.

Plots, figures

logger.save_fig(figure, name='foo')

Your figure should implement savefig interfaces from matplotlib. It's de facto standard for plots in Python.

  • name - denotes name of your file. File extensions are handled automatically.
logs/foo.png

Sheets

with open('foo.csv') as csv:
    frame = pd.read_csv(csv)
    logger.save_csv(frame, name='foo')

Your frame should implement to_csv interface from pandas. I guess everyone's just using pandas nowadays to handle sheets.

logs/foo.csv

Images

logger.save_image(image, name='astronaut')
logger.save_image(image, name='coffee')

We use skimage.io.imsave to handle images. Images are saved as png.

logs/astronaut.png
logs/coffee.png
logger.save_image(image, name='astronaut', prefix_step=True)
logger.save_image(image, name='coffee', prefix_step=True)

You can also use prefix_step parameter to append some ordering prefix. Guess now it's time to talk about steps. It's a private counter inside your logger, that's incremented each time you use log_func. So later, when you're viewing your files, you can order them by name (which is default for most OS). It'll result in following structure:

logs/01_astronaut.png
logs/02_coffee.png

Objects

logger.save_obj(logger, name='logger')

Objects are serialized using joblib.

Time to get a bit crazy (nesting)

with Logger(logs_dir) as logger:
    logger.add_entry('x', 0)

    with logger.get_child('bar') as child_logger:
        child_logger.add_entry('y', 1)

You can nest your loggers using get_child method. We recommend you wrap your loggers with scopes, so they're easier to manage.

  • once logger is disposed, it'll merge to the parent logger.
  • once root logger is disposed, it'll dump all your telemetry in json file

Another tip would be to write code in 'scope per branch' fashion - you do single method, where you use only single logger. You can pass parent logger with an argument.

Tests

cd [project-path]
python -m pytest .\tests\

Author

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for austen, version 0.1.6
Filename, size File type Python version Upload date Hashes
Filename, size austen-0.1.6-py3-none-any.whl (6.9 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size austen-0.1.6.tar.gz (5.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page