OpenLineage integration with dbt
Project description
OpenLineage dbt integration
Wrapper script for automatic metadata collection from dbt
Features
Metadata
- Model run lifecycle
- Model inputs / outputs
Requirements
Right now, openlineage-dbt supports only these dbt adapters:
bigquerysnowflakespark(thriftandodbc, but notlocal)redshiftathenagluepostgresclickhousetrinodatabrickssqlserverdremioduckdb
Installation
$ pip3 install openlineage-dbt
To install from source, run:
$ pip install .
Configuration
Transport
openlineage-dbt uses the OpenLineage Python client to push data to the OpenLineage backend, so any way of configuring Python client will work here as well:
OPENLINEAGE_URL=http://localhost:5000
OPENLINEAGE_API_KEY=abc
dbt integration-specific environment variables:
OPENLINEAGE_NAMESPACE- set if you are using something other than thedefaultnamespace for job namespace.
Logging
In addition to conventional logging approaches, the OpenLineage dbt wrapper script provides an alternative way of configuring its logging behavior. By setting the OPENLINEAGE_DBT_LOGGING environment variable, you can establish the logging level for the openlineage.dbt and its child modules.
You can also set log level of dbtol Python module but this is deprecated.
Usage
To begin collecting dbt metadata with OpenLineage, replace dbt run with dbt-ol run.
Additional table and column level metadata will be available if catalog.json, a result of running dbt docs generate, will be found in the target directory.
If you need to send events without running the job you can use the command dbt-ol send-events, it will send the metadata of your last run without running the job.
Development
For development setup instructions, see the Python development setup guide.
Quick start:
# From the repository root
$ make setup-dbt
# Or manually
$ cd integration/dbt
$ uv sync --extra dev
SPDX-License-Identifier: Apache-2.0
Copyright 2018-2026 contributors to the OpenLineage project
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file openlineage_dbt-1.44.1-py3-none-any.whl.
File metadata
- Download URL: openlineage_dbt-1.44.1-py3-none-any.whl
- Upload date:
- Size: 6.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.9.25
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
55d1075b54dbae5aa5382cc967fb4e18f54162978816375e8d02f04f0db715df
|
|
| MD5 |
43f430d379d182c44c1ddda4af6bec79
|
|
| BLAKE2b-256 |
42a8dbaf53a9d279da11d4fc6d1e7ad0887e7ba4cd46074f8ab70f28950de4b4
|