Versatile Data Kit SDK plugin that changes logging output to LTSV format.
Project description
This is a POC level implementation of a plugin which changes the logging format of vdk-core to LTSV for the purposes of structured data visualization.
The new format has the following fields, separated by tabs:
- timestamp: a timestamp of when the log is made;
- created: the same timestamp in the unix epoch format;
- jobname: the name of the data job;
- level: the logging level - one of INFO, WARNING, DEBUG, ERROR;
- modulename: the name of the module, inside of which the logging call is made;
- filename: the name of the file containing the logging call being made;
- lineno: the number of the line of code, at which the logging call is made;
- funcname: the name of the function, inside which the loggin call is made;
- attemptid: string identifying this particular execution of the data job;
- message: any additional logged information.
The label names follow the labelling recommendations found at http://ltsv.org/.
Usage
Switching vdk logging can be done by simply installing the plugin:
pip install vdk-logging-ltsv
And all logs will be automatically formatted to LTSV format. They would start to look something like this:
@timestamp:2021-08-04T12:51:11 created:1628070671 jobname:example-job level:DEBUG modulename:vdk.internal.trino_connection filename:managed_connection_base.py lineno:69 funcname:connect attemptid:1628070671-452613-739749 message:Established <trino.dbapi.Connection object at 0x10b9b1d30>
@timestamp:2021-08-04T12:51:11 created:1628070671 jobname:example-job level:DEBUG modulename:vdk.internal.trino_connection filename:managed_cursor.py lineno:29 funcname:execute attemptid:1628070671-452613-739749 message:Executing query: select 1
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for vdk-logging-ltsv-0.1.415625538.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | c56510f03b468840e172b35a4dbbd3d576ba1b87e48d6e555c6249d890a2a842 |
|
MD5 | db58611711d9fe1f7521f019df771d73 |
|
BLAKE2b-256 | 8113bf346ebd35517705f5198604a68e16f85713d4818e35c4a2b502501f8f0e |