Skip to main content

No project description provided

Project description


Quick Start:

This example Workflow File shows how to configure the Github Action via Github Actions Inputs.
This example Pull Request shows the output of the Github Action.
Screen Shot 2022-08-11 at 3 42 04 PM


Github Actions Input Arguments:

Input Parameter Description
GCP_TOKEN for connecting to BQ (runs dbt compile and dbt_table_diff/sql_checks to compare tables)
GH_TOKEN for connecting to Github (ie. fetches modified models/*.sql in your PR, adds comment on your PR)
PR_NUMBER for fetching open PR from github (Pull Request ID [int])
GH_REPO for fetching open PR from github (Repository Name)
GH_ORG for fetching open PR from github (Repository owner/organization name)
DBT_PROFILE_FILE the local path in your repo to your profile.yml for dbt (this is necessary for compiling manifest.json during setup process)
dev_prefix the prefix used when running dbt locally (Your source schema/environment for comparison)
prod_prefix the prefix used when running dbt remotely (Your target schema/environment for comparison)
fallback_prefix useful if you have an overriden macro for generate_schema_name in your dbt project, which leverages a different prefix for some schemas in prod.
irregular_schemas comma separated string of schemas which use fallback_prefix
project_id for connecting to BQ (BigQuery Project ID)
ignored_schemas comma separated string of schemas to ignore (skip checking during github action)

Description:

This Github Action is intended for comparing models in dbt that have changed during an open PR.
It only currently supports BigQuery.


Step-By-Step Break Down of Process:

  • Fetches list of files modified in Pull Request
    • by CURLing github.api.com/repos/{organization}/{repository}/pulls/{pull_request_id}/files
  • Filters on relevant_files
    • which are files matching models/*.sql
  • Builds manifest.json
    • By running dbt deps; dbt compile
  • Parses manifest.json for relevant_models
    • using manifest-attribute original_file_path matching relevant_files
  • Runs all SQL files in dbt_table_diff/sql_checks
    • for each of the relevant_models, compare the two dbt targets (dev_prefix vs prod_prefix)
  • Saves output to file
    • in a format supported by Github comments
  • Posts comment on open PR
    • leveraging py-github-helper PyPi package

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt_table_diff-2.2.0a1.tar.gz (18.8 kB view hashes)

Uploaded Source

Built Distribution

dbt_table_diff-2.2.0a1-py3-none-any.whl (19.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page