Parse structured metadata from dbt logs
Project description
dbt log parser
Python package and CLI tool for parsing structured (JSON) data from a dbt log.
Table of Contents
Generated Report
Example report structure:
{
"models_found": 469,
"tests_found": 1658,
"snapshots_found": 0,
"analyses_found": 0,
"macros_found": 300,
"operations_found": 0,
"seeds_found": 0,
"sources_found": 204,
"tests_run": 1100,
"tests_runtime_seconds": 97.47,
"tests": [
{
"number": 1,
"name": "accepted_values_dim_order_state_open",
"status": "PASS",
"total_time": 2.11
},
{
"status": "WARN",
"number": 2,
"name": "relationships_fact_contribution_goal_updates_contribution_page_id__contribution_page_id__ref_dim_contribution_page_",
"total_time": 1.73,
"query_results": {
"found": 5,
"expected": 0
},
"query": {
"filepath": "target/compiled/core/schema_test/relationships_fact_contribution_goal_updates_34dae512835158ed459182c173a8c127.sql",
"sql": null,
"file_err": true
}
},
{
"status": "FAIL",
"number": 548326,
"name": "unique_int_home_page_display_resources_you_may_like_event_user_id",
"total_time": 1.16,
"query_results": {
"found": 548326,
"expected": 0
},
"query": {
"filepath": "target/compiled/core/schema_test/unique_int_home_page_display_resources_you_may_like_event_user_id.sql",
"sql": null,
"file_err": true
}
}
]
}
See more in the JSON schema.
Usage
CLI
$ dbtlp --help
usage: dbtlp [-h] [--log-filepath LOG_FILEPATH] [--outfile OUTFILE]
DBT log parser
optional arguments:
-h, --help show this help message and exit
--log-filepath LOG_FILEPATH
Path to dbt log to parse
--outfile OUTFILE File to write JSON results to
$ dbtlp --log-filepath dbt.log --outfile results.json
API
from dbt_log_parser import parse
report = parse(log_filepath='dbt.log', write_report=False)
Contributing
See CONTRIBUTING.md
Change Log
Unreleased
0.1.0a0
- Initial release
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for dbt_log_parser-0.1.0a0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57cf5ed0163736d83a8a7e45c02f18ea7587655fb85db893482da5fca4496b0b |
|
MD5 | d908393d4f4a59dce02d8f26a84460db |
|
BLAKE2b-256 | 6564914df69ca0dd5223a0ce266eecc2b0052b06a4525122b76c4da9ce51051e |