Skip to main content

No project description provided

Project description


Quick Start:

This example Workflow File shows how to configure the Github Action via Github Actions Inputs.
This example Pull Request shows the output of the Github Action.
Screen Shot 2022-08-11 at 3 42 04 PM


Github Actions Input Arguments:

Input Parameter Description
GCP_TOKEN for connecting to BQ (runs dbt compile and dbt_table_diff/sql_checks to compare tables)
GH_TOKEN for connecting to Github (ie. fetches modified models/*.sql in your PR, adds comment on your PR)
PR_NUMBER for fetching open PR from github (Pull Request ID [int])
GH_REPO for fetching open PR from github (Repository Name)
GH_ORG for fetching open PR from github (Repository owner/organization name)
DBT_PROFILE_FILE the local path in your repo to your profile.yml for dbt (this is necessary for compiling manifest.json during setup process)
dev_prefix the prefix used when running dbt locally (Your source schema/environment for comparison)
prod_prefix the prefix used when running dbt remotely (Your target schema/environment for comparison)
fallback_prefix useful if you have an overriden macro for generate_schema_name in your dbt project, which leverages a different prefix for some schemas in prod.
irregular_schemas comma separated string of schemas which use fallback_prefix
project_id for connecting to BQ (BigQuery Project ID)
ignored_schemas comma separated string of schemas to ignore (skip checking during github action)

Description:

This Github Action is intended for comparing models in dbt that have changed during an open PR.
It only currently supports BigQuery.


Step-By-Step Break Down of Process:

  • Fetches list of files modified in Pull Request
    • by CURLing github.api.com/repos/{organization}/{repository}/pulls/{pull_request_id}/files
  • Filters on relevant_files
    • which are files matching models/*.sql
  • Builds manifest.json
    • By running dbt deps; dbt compile
  • Parses manifest.json for relevant_models
    • using manifest-attribute original_file_path matching relevant_files
  • Runs all SQL files in dbt_table_diff/sql_checks
    • for each of the relevant_models, compare the two dbt targets (dev_prefix vs prod_prefix)
  • Saves output to file
    • in a format supported by Github comments
  • Posts comment on open PR
    • leveraging py-github-helper PyPi package

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt_table_diff-2.2.0a1.tar.gz (18.8 kB view details)

Uploaded Source

Built Distribution

dbt_table_diff-2.2.0a1-py3-none-any.whl (19.5 kB view details)

Uploaded Python 3

File details

Details for the file dbt_table_diff-2.2.0a1.tar.gz.

File metadata

  • Download URL: dbt_table_diff-2.2.0a1.tar.gz
  • Upload date:
  • Size: 18.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for dbt_table_diff-2.2.0a1.tar.gz
Algorithm Hash digest
SHA256 92bce0bfc32c5b9d84ffc949923fe598b28d24df40254011c3cb45f0d360169b
MD5 1ea31180d2a26e0a88ca691f0e98e722
BLAKE2b-256 b5e76677808c317c7591476647bad4b5fb22576c5d05d06cc1cb0d6a2bbb8912

See more details on using hashes here.

File details

Details for the file dbt_table_diff-2.2.0a1-py3-none-any.whl.

File metadata

File hashes

Hashes for dbt_table_diff-2.2.0a1-py3-none-any.whl
Algorithm Hash digest
SHA256 c84d157ab3ecbdbe329f3f2df460e870b234380ff0c3da53a6f3d38d662d1478
MD5 a9219bfb2b1a8b3aca911e4d44a4c1a0
BLAKE2b-256 071dba2138ef29863b7dbd6ab0c179e43a189fb51453a69490aa282b479019bf

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page