Structlog extensions are a set of structlog processors and utility functions to add new logging options to structlog. The primary purpose is to supply tools to convert existing structlog logging output into Elastic Common Schema json output so users can easily plug their application output into centralised logging solutions such as ELK stack.
At present the extensions consist of a CombinedLogParser (which for example can be used to convert gunicorn access log output into ECS fields), and the NestedDictJSONRenderer, which can be used to convert the output of the CombinedLogParser into ECS json format output.
Usage
CombinedLogParser
This processor will parse events formatted in Apache Combined log format into Elastic common schema fields.
Example
This is an example for configuring gunicorn to emit json logs.
gunicorn.conf.py
import structlog
import structlog_extensions
# --- Structlog logging initialisation code
pre_chain = [
# Add the log level and a timestamp to the event_dict if the log entry
# is not from structlog.
structlog.stdlib.add_log_level,
structlog.stdlib.add_logger_name,
structlog_extensions.processors.CombinedLogParser("gunicorn.access")
]
logconfig_dict = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"json_formatter": {
"()": structlog.stdlib.ProcessorFormatter,
"processor": structlog.processors.JSONRenderer(),
"foreign_pre_chain": pre_chain,
}
},
"handlers": {
"error_console": {
"class": "logging.StreamHandler",
"formatter": "json_formatter",
},
"console": {
"class": "logging.StreamHandler",
"formatter": "json_formatter",
}
},
}
NestedDictJSONRenderer
This processor will convert key names using a specified separator into nested dictionaries prior to rendering the output as JSON using the structlog JSONRenderer. This processor can for example convert Elastic Common Schema namespaced keynames produced by the CombinedLogParser into the nested JSON format that ECS specifies. This processor must be the final processor in a chain because it renders the output as a string instead of passing along an event dictionary.
Example
When using this logging initialisation:
# --- std logging initialisation code using structlog rendering import structlog import structlog_extensions pre_chain = [ # Add the log level and a timestamp to the event_dict if the log entry # is not from structlog. structlog.stdlib.add_log_level, structlog.stdlib.add_logger_name, structlog_extensions.processors.CombinedLogParser("gunicorn.access") ] logging.dict_config( { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog_extensions.processors.NestedDictJSONRenderer('.'), "foreign_pre_chain": pre_chain, } }, "handlers": { "error_console": { "class": "logging.StreamHandler", "formatter": "json_formatter", }, "console": { "class": "logging.StreamHandler", "formatter": "json_formatter", } }, })These entries (produced by structlog_extensions.processors.CombinedLogParser):
{ 'http.request.method': 'get', 'http:.request.referrer': 'http://www.example.com', 'http.version': '1.0'}`will be transformed into the following nested json structure:
{ 'http': { 'version': '1.0', 'request': { 'method': 'get', 'referrer': 'http://www.example.com'} } }
Metadata
Release files for structlog-extensions-nralbers 1.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| structlog-extensions-nralbers-1.0.4.tar.gz | 8.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| structlog_extensions_nralbers-1.0.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 16.9 kB
Release files / structlog-extensions-nralbers-1.0.4.tar.gz
| Download URL | structlog-extensions-nralbers-1.0.4.tar.gz |
|---|---|
| Size | 8.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5a203574a72b733cda050390ebd7e209bcca25c295ed3c3b40e486a51eb6bf4c
|
|
BLAKE2b-256 checksum How to use checksums |
1f5eb738b96e30aba54a48025b82bef914c953a5441fcee51d353cb80b8fbedd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.8.5
|
Release files / structlog_extensions_nralbers-1.0.4-py3-none-any.whl
| Download URL | structlog_extensions_nralbers-1.0.4-py3-none-any.whl |
|---|---|
| Size | 8.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
264fee9cca2d7f47a71583ee3b69fa7e1bd524d37a1a974a0a3ffc0e3f011158
|
|
BLAKE2b-256 checksum How to use checksums |
7dd63a03b82a14b0b4f51e2bdae7603471aa2fbf5eaf2a2cd300f3f47f984a62
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.5.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.8.5
|