Skip to main content

The Marquez BigQuery adapter plugin for dbt (data build tool)

Project description

dbt-bigquery-openlineage

Plugin for dbt (data build tool) for automatic metadata and lineage collection.

This integration is experimental, and there is high possibility that it will change in the future, as together with dbt developers we're figuring what's the best way to extract metadata from it.

Requirements

Python 3.6+
sqlparse>=0.3.1

Installation

$ pip install marquez-dbt-bigquery

Configuration

You need to configure dbt's profiles.yaml file. This plugin introduces additional type: openlineage_bigquery.

It handles the same configuration flags as original bigquery plugin, with the addition of OpenLineage related ones:

    openlineage_url - sets URL of OpenLineage server where events will be pushed
    openlineage_timeout - optional - sets timeout for connecting to OpenLineage server. By default it's set to 5 seconds.
    openlineage_api_key - optional - sets api key for OpenLineage server

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

marquez-dbt-bigquery-0.16.1.tar.gz (5.9 kB view details)

Uploaded Source

Built Distribution

marquez_dbt_bigquery-0.16.1-py3-none-any.whl (11.6 kB view details)

Uploaded Python 3

File details

Details for the file marquez-dbt-bigquery-0.16.1.tar.gz.

File metadata

  • Download URL: marquez-dbt-bigquery-0.16.1.tar.gz
  • Upload date:
  • Size: 5.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.6.14

File hashes

Hashes for marquez-dbt-bigquery-0.16.1.tar.gz
Algorithm Hash digest
SHA256 165a1b4eada8a1077b06f5493cf8ca465af39d40c1fef6cbfcd27b84dad5e4d9
MD5 cbde9295a2b5d0e888e6e095db55bbf7
BLAKE2b-256 23611dfeb88fd1bab6bc4855a25da7157f0fe757ac03d504ccadbda996163ac7

See more details on using hashes here.

File details

Details for the file marquez_dbt_bigquery-0.16.1-py3-none-any.whl.

File metadata

  • Download URL: marquez_dbt_bigquery-0.16.1-py3-none-any.whl
  • Upload date:
  • Size: 11.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.6.14

File hashes

Hashes for marquez_dbt_bigquery-0.16.1-py3-none-any.whl
Algorithm Hash digest
SHA256 3e91b8e2a28d29a9189bc98304c5b5a947639120cde0dba5ab0ef8355f3c2f0a
MD5 7cea1df44884f351bc2ca04192b9ee75
BLAKE2b-256 33f055289b448ba0af7f0b392fe6cfadb481cd7643428d3b5a43f8ab57e019a0

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page