Versatile Data Kit SDK plugin that changes logging output to LTSV format.
Project description
This is a POC level implementation of a plugin which changes the logging format of vdk-core to LTSV for the purposes of structured data visualization.
The new format has the following fields, separated by tabs:
- timestamp: a timestamp of when the log is made;
- created: the same timestamp in the unix epoch format;
- jobname: the name of the data job;
- level: the logging level - one of INFO, WARNING, DEBUG, ERROR;
- modulename: the name of the module, inside of which the logging call is made;
- filename: the name of the file containing the logging call being made;
- lineno: the number of the line of code, at which the logging call is made;
- funcname: the name of the function, inside which the loggin call is made;
- attemptid: string identifying this particular execution of the data job;
- message: any additional logged information.
The label names follow the labelling recommendations found at http://ltsv.org/.
Usage
Switching vdk logging can be done by simply installing the plugin:
pip install vdk-logging-ltsv
And all logs will be automatically formatted to LTSV format. They would start to look something like this:
@timestamp:2021-08-04T12:51:11 created:1628070671 jobname:example-job level:DEBUG modulename:taurus.vdk.trino_connection filename:managed_connection_base.py lineno:69 funcname:connect attemptid:1628070671-452613-739749 message:Established <trino.dbapi.Connection object at 0x10b9b1d30>
@timestamp:2021-08-04T12:51:11 created:1628070671 jobname:example-job level:DEBUG modulename:taurus.vdk.trino_connection filename:managed_cursor.py lineno:29 funcname:execute attemptid:1628070671-452613-739749 message:Executing query: select 1
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for vdk-logging-ltsv-0.1.353684692.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9a99cef95737d3c6617488a749f7a0af6aec164d02f08be931828369ea99714c |
|
MD5 | 78ad24cf31cab8b20f1a498534e4b87b |
|
BLAKE2b-256 | 1dc805800787cef94508ba4a9f4deab49cfdafa9ee41933d6546a51df0f8c6b8 |