Skip to main content

Structlog extensions are a set of structlog processors and utility functions to add new logging options to structlog. The primary purpose is to supply tools to convert existing structlog logging output into Elastic Common Schema json output so users can easily plug their application output into centralised logging solutions such as ELK stack.

At present the extensions consist of a CombinedLogParser (which for example can be used to convert gunicorn access log output into ECS fields), and the NestedDictJSONRenderer, which can be used to convert the output of the CombinedLogParser into ECS json format output.

Usage

CombinedLogParser

This processor will parse events formatted in Apache Combined log format into Elastic common schema fields.

Example

This is an example for configuring gunicorn to emit json logs.

gunicorn.conf.py

import structlog
import structlog_extensions

# --- Structlog logging initialisation code

pre_chain = [
    # Add the log level and a timestamp to the event_dict if the log entry
    # is not from structlog.
    structlog.stdlib.add_log_level,
    structlog.stdlib.add_logger_name,
    structlog_extensions.processors.CombinedLogParser("gunicorn.access")
]

logconfig_dict = {
            "version": 1,
            "disable_existing_loggers": False,
            "formatters": {
                "json_formatter": {
                    "()": structlog.stdlib.ProcessorFormatter,
                    "processor": structlog.processors.JSONRenderer(),
                    "foreign_pre_chain": pre_chain,
                }
            },
            "handlers": {
                "error_console": {
                    "class": "logging.StreamHandler",
                    "formatter": "json_formatter",
                },
                "console": {
                    "class": "logging.StreamHandler",
                    "formatter": "json_formatter",
                }
            },
        }

NestedDictJSONRenderer

This processor will convert key names using a specified separator into nested dictionaries prior to rendering the output as JSON using the structlog JSONRenderer. This processor can for example convert Elastic Common Schema namespaced keynames produced by the CombinedLogParser into the nested JSON format that ECS specifies. This processor must be the final processor in a chain because it renders the output as a string instead of passing along an event dictionary.

Example

When using this logging initialisation:

# --- std logging initialisation code using structlog rendering
import structlog
import structlog_extensions

pre_chain = [
    # Add the log level and a timestamp to the event_dict if the log entry
    # is not from structlog.
    structlog.stdlib.add_log_level,
    structlog.stdlib.add_logger_name,
    structlog_extensions.processors.CombinedLogParser("gunicorn.access")
]

logging.dict_config( {
            "version": 1,
            "disable_existing_loggers": False,
            "formatters": {
                "json_formatter": {
                    "()": structlog.stdlib.ProcessorFormatter,
                    "processor": structlog_extensions.processors.NestedDictJSONRenderer('.'),
                    "foreign_pre_chain": pre_chain,
                }
            },
            "handlers": {
                "error_console": {
                    "class": "logging.StreamHandler",
                    "formatter": "json_formatter",
                },
                "console": {
                    "class": "logging.StreamHandler",
                    "formatter": "json_formatter",
                }
            },
        })

These entries (produced by structlog_extensions.processors.CombinedLogParser):

{ 'http.request.method': 'get', 'http:.request.referrer': 'http://www.example.com', 'http.version': '1.0'}`

will be transformed into the following nested json structure:

{ 'http': { 'version': '1.0',
            'request': { 'method': 'get',
                         'referrer': 'http://www.example.com'}
            }
}

Metadata

Release files for structlog-extensions-nralbers 1.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for structlog-extensions-nralbers 1.0.4
File Size Uploaded
structlog-extensions-nralbers-1.0.4.tar.gz 8.3 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for structlog-extensions-nralbers 1.0.4
File Interpreter ABI Platform
structlog_extensions_nralbers-1.0.4-py3-none-any.whl Python 3 none any Details

Total release size: 16.9 kB

Release files / structlog-extensions-nralbers-1.0.4.tar.gz

Download URL structlog-extensions-nralbers-1.0.4.tar.gz
Size 8.3 kB
Tags Source
SHA-256 checksum
How to use checksums
5a203574a72b733cda050390ebd7e209bcca25c295ed3c3b40e486a51eb6bf4c
BLAKE2b-256 checksum
How to use checksums
1f5eb738b96e30aba54a48025b82bef914c953a5441fcee51d353cb80b8fbedd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.8.5

Release files / structlog_extensions_nralbers-1.0.4-py3-none-any.whl

Download URL structlog_extensions_nralbers-1.0.4-py3-none-any.whl
Size 8.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
264fee9cca2d7f47a71583ee3b69fa7e1bd524d37a1a974a0a3ffc0e3f011158
BLAKE2b-256 checksum
How to use checksums
7dd63a03b82a14b0b4f51e2bdae7603471aa2fbf5eaf2a2cd300f3f47f984a62
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.8.5

Release history Release notifications | RSS feed

This release

1.0.4 This release

2 release files

1.0.3

2 release files

1.0.2

2 release files

1.0.1

2 release files

1.0.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page