No project description provided
Project description
Quick Start:
This example Workflow File shows how to configure the Github Action via Github Actions Inputs
.
This example Pull Request shows the output of the Github Action.
Github Actions Input Arguments:
Input Parameter | Description |
---|---|
GCP_TOKEN | for connecting to BQ (runs dbt compile and dbt_table_diff/sql_checks to compare tables) |
GH_TOKEN | for connecting to Github (ie. fetches modified models/*.sql in your PR, adds comment on your PR) |
PR_NUMBER | for fetching open PR from github (Pull Request ID [int]) |
GH_REPO | for fetching open PR from github (Repository Name) |
GH_ORG | for fetching open PR from github (Repository owner/organization name) |
DBT_PROFILE_FILE | the local path in your repo to your profile.yml for dbt (this is necessary for compiling manifest.json during setup process) |
dev_prefix | the prefix used when running dbt locally (Your source schema/environment for comparison) |
prod_prefix | the prefix used when running dbt remotely (Your target schema/environment for comparison) |
fallback_prefix | useful if you have an overriden macro for generate_schema_name in your dbt project, which leverages a different prefix for some schemas in prod. |
irregular_schemas | comma separated string of schemas which use fallback_prefix |
project_id | for connecting to BQ (BigQuery Project ID) |
ignored_schemas | comma separated string of schemas to ignore (skip checking during github action) |
Description:
This Github Action
is intended for comparing models
in dbt
that have changed during an open PR.
It only currently supports BigQuery
.
Step-By-Step Break Down of Process:
- Fetches list of files modified in Pull Request
- by CURLing
github.api.com/repos/{organization}/{repository}/pulls/{pull_request_id}/files
- by CURLing
- Filters on
relevant_files
- which are files matching
models/*.sql
- which are files matching
- Builds
manifest.json
- By running
dbt deps; dbt compile
- By running
- Parses
manifest.json
forrelevant_models
- using manifest-attribute
original_file_path
matchingrelevant_files
- using manifest-attribute
- Runs all SQL files in
dbt_table_diff/sql_checks
- for each of the
relevant_models
, compare the two dbt targets (dev_prefix
vsprod_prefix
)
- for each of the
- Saves output to file
- in a format supported by Github comments
- Posts comment on open PR
- leveraging
py-github-helper
PyPi package
- leveraging
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dbt_table_diff-2.2.0a1.tar.gz
(18.8 kB
view details)
Built Distribution
File details
Details for the file dbt_table_diff-2.2.0a1.tar.gz
.
File metadata
- Download URL: dbt_table_diff-2.2.0a1.tar.gz
- Upload date:
- Size: 18.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.9.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 92bce0bfc32c5b9d84ffc949923fe598b28d24df40254011c3cb45f0d360169b |
|
MD5 | 1ea31180d2a26e0a88ca691f0e98e722 |
|
BLAKE2b-256 | b5e76677808c317c7591476647bad4b5fb22576c5d05d06cc1cb0d6a2bbb8912 |
File details
Details for the file dbt_table_diff-2.2.0a1-py3-none-any.whl
.
File metadata
- Download URL: dbt_table_diff-2.2.0a1-py3-none-any.whl
- Upload date:
- Size: 19.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.9.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c84d157ab3ecbdbe329f3f2df460e870b234380ff0c3da53a6f3d38d662d1478 |
|
MD5 | a9219bfb2b1a8b3aca911e4d44a4c1a0 |
|
BLAKE2b-256 | 071dba2138ef29863b7dbd6ab0c179e43a189fb51453a69490aa282b479019bf |