The Marquez BigQuery adapter plugin for dbt (data build tool)
Project description
dbt-bigquery-openlineage
Plugin for dbt (data build tool) for automatic metadata and lineage collection.
This integration is experimental, and there is high possibility that it will change in the future, as together with dbt developers we're figuring what's the best way to extract metadata from it.
Requirements
Python 3.6+
sqlparse>=0.3.1
Installation
$ pip install marquez-dbt-bigquery
Configuration
You need to configure dbt's profiles.yaml file. This plugin introduces additional type: openlineage_bigquery.
It handles the same configuration flags as original bigquery plugin, with the addition of OpenLineage related ones:
openlineage_url - sets URL of OpenLineage server where events will be pushed
openlineage_timeout - optional - sets timeout for connecting to OpenLineage server. By default it's set to 5 seconds.
openlineage_api_key - optional - sets api key for OpenLineage server
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file marquez-dbt-bigquery-0.16.1.tar.gz
.
File metadata
- Download URL: marquez-dbt-bigquery-0.16.1.tar.gz
- Upload date:
- Size: 5.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.6.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 165a1b4eada8a1077b06f5493cf8ca465af39d40c1fef6cbfcd27b84dad5e4d9 |
|
MD5 | cbde9295a2b5d0e888e6e095db55bbf7 |
|
BLAKE2b-256 | 23611dfeb88fd1bab6bc4855a25da7157f0fe757ac03d504ccadbda996163ac7 |
File details
Details for the file marquez_dbt_bigquery-0.16.1-py3-none-any.whl
.
File metadata
- Download URL: marquez_dbt_bigquery-0.16.1-py3-none-any.whl
- Upload date:
- Size: 11.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.6.1 pkginfo/1.7.1 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.2 CPython/3.6.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3e91b8e2a28d29a9189bc98304c5b5a947639120cde0dba5ab0ef8355f3c2f0a |
|
MD5 | 7cea1df44884f351bc2ca04192b9ee75 |
|
BLAKE2b-256 | 33f055289b448ba0af7f0b392fe6cfadb481cd7643428d3b5a43f8ab57e019a0 |