OpenLineage integration with dbt
Project description
OpenLineage dbt integration
Wrapper script for dbt for automatic metadata collection
Features
Metadata
- Model run lifecycle
- Model inputs / outputs
Requirements
Installation
$ pip3 install openlineage-dbt
To install from source, run:
$ pip install .
Configuration
HTTP
Backend Environment Variables
openlineage-dbt
uses OpenLineage client to push data to OpenLineage backend.
OpenLineage client depends on environment variables:
OPENLINEAGE_URL
- point to service which will consume OpenLineage eventsOPENLINEAGE_API_KEY
- set if consumer of OpenLineage events requiresBearer
authentication keyOPENLINEAGE_NAMESPACE
- set if you are using something other than thedefault
namespace for job namespace.
Usage
To begin collecting dbt metadata with OpenLineage, replace dbt run
with dbt-ol run
.
Additional table and column level metadata will be available if catalog.json
, result of running dbt docs generate
will be found in target directory.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
openlineage-dbt-0.2.3.tar.gz
(2.9 kB
view hashes)
Built Distribution
Close
Hashes for openlineage_dbt-0.2.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 055e4212ac778ac84bc8b27f12a78021b963222a58ef4cc74d3ed35fbbcc311e |
|
MD5 | 7fc3baf9bbf74707d8a4c3cc11d5429d |
|
BLAKE2b-256 | 9623f0fb0b14ed3d4402c09d0174d2214fcd334ad9e0fbe0531ecadfe1b1ad01 |