Skip to main content

CLI tool for managing + visualising BigQuery authorised views

Project description

BigQuery View Analyzer

PyPI version Python versions Build status Github license

Description

bigquery-view-analyzer is a command-line tool for visualizing dependencies and managing permissions between BigQuery views.

To authorize a view, permissions must be granted at a dataset level for every view/table referenced in the view definition. This requirement cascades down to every view that's referenced by the parent view, they too must have permissions granted for every view/table they reference - and so on. This can quickly become difficult to manage if you have many nested views across multiple datasets and/or projects.

bigquery-view-analyzer automatically resolves these dependencies and applies the relevant permissions to all views and datasets referenced by the parent view.

Installation

$ pip install bigquery-view-analyzer

Usage

$ bqva --help

asciicast

Example: CLI

Example tree

Given the above datasets and tables in BigQuery, to authorize bqva-demo:dataset_4.shared_view, the following views would need to be authorized with each of the following datasets:

  • Authorized views for dataset_1
    • bqva-demo:dataset_3.view_a_b_c_d
  • Authorized views for dataset_2
    • bqva-demo:dataset_3.view_a_b_c_d
    • bqva-demo:dataset_1.view_c
  • Authorized views for dataset_3
    • bqva-demo:dataset_2.view_d
    • bqva-demo:dataset_4.shared_view

You can easily visualize the above view hierarchy using the bqva tree command.

# View dependency tree and authorization status for 'bqva-demo:dataset_4.shared_view'
$ bqva tree --status --no-key --view "bqva-demo:dataset_4.shared_view"
bqva-demo:dataset_4.shared_view
└── bqva-demo:dataset_3.view_a_b_c_d ()
    ├── bqva-demo:dataset_1.table_a ()
    ├── bqva-demo:dataset_1.table_b ()
    ├── bqva-demo:dataset_1.view_c ()
       └── bqva-demo:dataset_2.table_c ()
    └── bqva-demo:dataset_2.view_d ()
        └── bqva-demo:dataset_3.table_d ()

Permissions can be applied automatically to all datasets referenced by the parent view using the bqva authorize command.

# Apply all permissions required by 'bqva-demo:dataset_4.shared_view'
$ bqva authorize --view "bqva-demo:dataset_4.shared_view"
bqva-demo:dataset_4.shared_view
└── bqva-demo:dataset_3.view_a_b_c_d ()
    ├── bqva-demo:dataset_1.table_a ()
    ├── bqva-demo:dataset_1.table_b ()
    ├── bqva-demo:dataset_1.view_c ()
       └── bqva-demo:dataset_2.table_c ()
    └── bqva-demo:dataset_2.view_d ()
        └── bqva-demo:dataset_3.table_d ()

If you want to revoke permissions for a view, you can do that too!

# Revoke all permissions granted to 'bqva-demo:dataset_4.shared_view'
$ bqva revoke --view "bqva-demo:dataset_4.shared_view"
bqva-demo:dataset_4.shared_view
└── bqva-demo:dataset_3.view_a_b_c_d ()
    ├── bqva-demo:dataset_1.table_a ()
    ├── bqva-demo:dataset_1.table_b ()
    ├── bqva-demo:dataset_1.view_c ()
       └── bqva-demo:dataset_2.table_c ()
    └── bqva-demo:dataset_2.view_d ()
        └── bqva-demo:dataset_3.table_d ()

Example: Python library

You can import the library within a Python project to programatically apply permissions to multiple datasets.

from bqva import ViewAnalyzer
from google.cloud import bigquery

client = bigquery.Client()


def auth_views(datasets=[], **kwargs):
    # get all datasets by default if none provided
    if len(datasets) == 0:
        datasets = client.list_datasets(max_results=1)
    for dataset in datasets:
        dataset = client.dataset(dataset)
        tables = client.list_tables(dataset.dataset_id)
        for table in tables:
            if table.table_type == "VIEW":
                view = ViewAnalyzer(
                    project_id=table.project,
                    dataset_id=table.dataset_id,
                    view_id=table.table_id,
                )
                view.apply_permissions()
            print(
                f"Authorised view: {table.project}.{table.dataset_id}.{table.table_id}"
            )


auth_views(["dataset_a", "dataset_b"])

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bigquery-view-analyzer-21.9.0.tar.gz (7.7 kB view details)

Uploaded Source

File details

Details for the file bigquery-view-analyzer-21.9.0.tar.gz.

File metadata

  • Download URL: bigquery-view-analyzer-21.9.0.tar.gz
  • Upload date:
  • Size: 7.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.8 CPython/3.7.7 Linux/5.8.0-1035-gcp

File hashes

Hashes for bigquery-view-analyzer-21.9.0.tar.gz
Algorithm Hash digest
SHA256 92b70ce835caea53d1f04c984332f6344357219fb33f411fc2c7c6b35d0f7dd8
MD5 c167c6d50712973c60bb61fde651d92d
BLAKE2b-256 89eb0a86f91db64435b93f0d21d188f85e3c405141e6caf8fec8f9628f6b6e40

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page