Skip to main content

OpenLineage integration with dbt

Project description

OpenLineage dbt integration

Wrapper script for automatic metadata collection from dbt

Features

Metadata

  • Model run lifecycle
  • Model inputs / outputs

Requirements

Right now, openlineage-dbt supports only these dbt adapters:

  • bigquery
  • snowflake
  • spark (thrift and odbc, but not local)
  • redshift
  • athena
  • glue
  • postgres
  • clickhouse
  • trino
  • databricks
  • sqlserver
  • dremio
  • duckdb

Installation

$ pip3 install openlineage-dbt

To install from source, run:

$ pip install .

Configuration

Transport

openlineage-dbt uses the OpenLineage Python client to push data to the OpenLineage backend, so any way of configuring Python client will work here as well:

OPENLINEAGE_URL=http://localhost:5000
OPENLINEAGE_API_KEY=abc

dbt integration-specific environment variables:

  • OPENLINEAGE_NAMESPACE - set if you are using something other than the default namespace for job namespace.

Logging

In addition to conventional logging approaches, the OpenLineage dbt wrapper script provides an alternative way of configuring its logging behavior. By setting the OPENLINEAGE_DBT_LOGGING environment variable, you can establish the logging level for the openlineage.dbt and its child modules.

You can also set log level of dbtol Python module but this is deprecated.

Usage

To begin collecting dbt metadata with OpenLineage, replace dbt run with dbt-ol run.

Additional table and column level metadata will be available if catalog.json, a result of running dbt docs generate, will be found in the target directory.

If you need to send events without running the job you can use the command dbt-ol send-events, it will send the metadata of your last run without running the job.

Development

For development setup instructions, see the Python development setup guide.

Quick start:

# From the repository root
$ make setup-dbt

# Or manually
$ cd integration/dbt
$ uv sync --extra dev

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2026 contributors to the OpenLineage project

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

openlineage_dbt-1.44.1-py3-none-any.whl (6.3 kB view details)

Uploaded Python 3

File details

Details for the file openlineage_dbt-1.44.1-py3-none-any.whl.

File metadata

File hashes

Hashes for openlineage_dbt-1.44.1-py3-none-any.whl
Algorithm Hash digest
SHA256 55d1075b54dbae5aa5382cc967fb4e18f54162978816375e8d02f04f0db715df
MD5 43f430d379d182c44c1ddda4af6bec79
BLAKE2b-256 42a8dbaf53a9d279da11d4fc6d1e7ad0887e7ba4cd46074f8ab70f28950de4b4

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page