Overview

This application is used to perform a BigQuery dry run of all .sql files in a folder and its subfolders.

Its purpose it so ensure that all SQL files do not have any syntactic errors and make it easier for team members to quickly check a large number of files.

GCP configuration

Allowing access to GCP should be done following one of the methods detailed in the GCP documentation.

The application will rely on your environment having been set up for authentication to GCP (e.g. through gcloud init or environement variable containing service account credentials), it does not provide a mechanism to receive authentication credentials directly.

Usage

The application will install a shell command called bqdry which is simply passed a folder. It will then traverse the folder and perform a dry run of all .sql files found in the folder and any sub folders. The results will be displayed on in the terminal.

For example:

$ bqdry my-awesome-project

>File: my-awesome-project/demo.sql
>  Result: Failed
>  Errors: None
>
>Total: 3
>Succeeded: 2
>Failed: 1

bqdry -h will provide usage information. E.g.

usage: bqdry [-h] [-t THREADS] [-v] folder

Dry run of all `.sql` files in folder and subfolders.

positional arguments:
  folder                Top level folder to start scanning for `.sql` files.

optional arguments:
  -h, --help            show this help message and exit
  -t THREADS, --threads THREADS
                        Number of threads for concurrent running of queries. Defaults to 2.
  -v, --verbose         Show all file results, not just failures.