Skip to main content

A simple library to emit contextual information in structured logs (JSON)

Project description

Python context-log library


context-log is a simple library to emit contextual information in structured logs (JSON).

It works particularly well in a Docker or Serverless (e.g. AWS Lambda) environment where a single thread executes a request and produces a response.

The library uses python threading to store contextual information that is automatically added to all subsequent logs in a contextMap field.

Because the library uses the Python thread local context it works across packages and modules in a given project.

The approach is loosely based on the Log4j 2 API Thread Context.


Structured logging can be achieved with the python-json-logger library.

Simply add project dependencies to requirements.txt:


Add the code below to the main code module.

Add the following YAML configuration in the resources/logging.yaml file, which outputs JSON structured logs to stdout.

version: 1
    class:  .jsonlogger.JsonFormatter
    format: '%(asctime)s %(name)s %(levelname)s %(message)s %(filename)s'
    class: logging.StreamHandler
    level: DEBUG
    formatter: json
  level: DEBUG
    - console

Use the context_log library to emit logs. Example below.

import logging.config
import yaml

with open('resources/logging.yaml', 'r') as log_config_file:

from context_log import ContextLog

def handler(event, context):
    # Clear context (e.g. re-use) and get logger
    log = ContextLog.get_logger('handler', True)'start')

    ContextLog.put('ip', '')

    # Helper to add start time in ISO and epoch time

    # Process request

    # Helper to add end time in ISO and epoch time
    # as well as duration in milliseconds

First log info event:

    "asctime": "2019-09-19 11:53:20,479",
    "name": "handler",
    "levelname": "INFO",
    "message": "start",
    "filename": "",
    "contextMap": {}

Second log info event:

  "asctime": "2019-09-19 11:53:20,580",
  "name": "handler",
  "levelname": "INFO",
  "message": "end",
  "filename": "",
  "contextMap": {
      "ip": "",
      "start-time": "2019-09-19T11:53:20.480085",
      "epoch-start-time": 1568890400.480085,
      "end-time": "2019-09-19T11:53:20.580513",
      "epoch-end-time": 1568890400.580513,
      "duration": 100.428}

The Detail

The standard logger is wrapped by a LoggerAdapter. It is therefore imperative that the ContextLog.get_logger(name='<name>', clear=True|False) call is made to get the logger to emit contextual logs.

Use clear=True when starting a new request in order to clear the previous context if the thread is re-used. This is typically the case in thread pools and in AWS Lambda's.

To manipulate or retrieve the contextMap use the following methods:

  • clear()
  • put(key, value)
  • get(key)
  • get_map()

There are also a number of helpers in an attempt to standardise log output contextMap fields:

  • put_request_id(request_id)
  • put_request_method(request_method)
  • put_request_path(request_path)
  • put_response_status(response_status)
  • put_start_time()
  • put_end_time()
  • put_request_user_id(request_user_id)
  • put_request_client_id(request_client_id)
  • put_request_primary_ip(primary_ip)
  • put_request_client_ip(client_ip)
  • put_request_viewer_country(viewer_country)
  • put_trigger_source(trigger_source)


Pull requests are more than welcome.

Running pytest

Create virtualenv, download dependencies and run tests:

python3 -m venv .venv
source .venv/bin/activate
pip3 install -r tests/requirements.txt
pip3 install -e .

Running tox

pip3 install --user --upgrade tox

Releasing library to PyPI

Short version from the Packaging Python Projects site.

Install the release tools:

python3 -m pip install --user --upgrade setuptools wheel twine

Remove old distribution(s):

rm -rf dist/

Build the context-log package:

python3 sdist bdist_wheel

Upload context-log first to Test PyPI:

python3 -m twine upload --repository-url dist/*

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

context-log-0.1.2.tar.gz (4.8 kB view hashes)

Uploaded source

Built Distribution

context_log-0.1.2-py3-none-any.whl (6.0 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page