Versatile Data Kit SDK plugin that changes logging output to LTSV format.
Project description
This is a POC level implementation of a plugin which changes the logging format of vdk-core to LTSV for the purposes of structured data visualization.
The new format has the following fields, separated by tabs:
- timestamp: a timestamp of when the log is made;
- created: the same timestamp in the unix epoch format;
- jobname: the name of the data job;
- level: the logging level - one of INFO, WARNING, DEBUG, ERROR;
- modulename: the name of the module, inside of which the logging call is made;
- filename: the name of the file containing the logging call being made;
- lineno: the number of the line of code, at which the logging call is made;
- funcname: the name of the function, inside which the loggin call is made;
- attemptid: string identifying this particular execution of the data job;
- message: any additional logged information.
The label names follow the labelling recommendations found at http://ltsv.org/.
Usage
Switching vdk logging can be done by simply installing the plugin:
pip install vdk-logging-ltsv
And all logs will be automatically formatted to LTSV format. They would start to look something like this:
@timestamp:2021-08-04T12:51:11 created:1628070671 jobname:example-job level:DEBUG modulename:taurus.vdk.trino_connection filename:managed_connection_base.py lineno:69 funcname:connect attemptid:1628070671-452613-739749 message:Established <trino.dbapi.Connection object at 0x10b9b1d30>
@timestamp:2021-08-04T12:51:11 created:1628070671 jobname:example-job level:DEBUG modulename:taurus.vdk.trino_connection filename:managed_cursor.py lineno:29 funcname:execute attemptid:1628070671-452613-739749 message:Executing query: select 1
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for vdk-logging-ltsv-0.1.348963555.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 98bda9c6da8609bef32a2bda24fbfe5bf5a79a3b8c51614741eaadff34407569 |
|
MD5 | 6142ecc44c9c98b598d33ea5d7bff732 |
|
BLAKE2b-256 | 64982aa0b0f71e9007ed32e13d1add6c2ddcb00c8396f80ff2b11f4664e994fc |