Skip to main content

The trino adapter plugin for dbt (data build tool)

Project description

dbt-trino

Starburst dbt         trino

Build Status db-starburst-and-trino Slack

Introduction

dbt is a data transformation workflow tool that lets teams quickly and collaboratively deploy analytics code, following software engineering best practices like modularity, CI/CD, testing, and documentation. It enables anyone who knows SQL to build production-grade data pipelines.

One frequently asked question in the context of using dbt tool is:

Can I connect my dbt project to two databases?

(see the answered question on the dbt website).

TL;DR dbt stands for transformation as in T within ELT pipelines, it doesn't move data from source to a warehouse.

dbt-trino adapter uses Trino as a underlying query engine to perform query federation across disperse data sources. Trino connects to multiple and diverse data sources (available connectors) via one dbt connection and process SQL queries at scale. Transformations defined in dbt are passed to Trino which handles these SQL transformation queries and translates them to queries specific to the systems it connects to create tables or views and manipulate data.

This repository represents a fork of the dbt-presto with adaptations to make it work with Trino.

Compatibility

This dbt plugin has been tested against Trino version 475, Starburst Enterprise version 468-e.6 and Starburst Galaxy.

Setup & Configuration

For information on installing and configuring your profile to authenticate to Trino or Starburst, please refer to Starburst and Trino Setup in the dbt docs.

Trino- and Starburst-specific configuration

For Trino- and Starburst-specific configuration, you can refer to Starburst (Trino) configurations on the dbt docs site.

Contributing

Release process

First 5 steps are ONLY relevant for bumping minor version:

  1. Create 1.x.latest branch from the latest tag corresponding to current minor version, e.g. git checkout -b 1.6.latest v1.6.2 (when bumping to 1.7). Push branch to remote. This branch will be used for potential backports.
  2. Create new branch (Do not push below commits to 1.x.latest). Add a new entry in .changes/0.0.0.md that points to the newly created latest branch.
  3. Run changie merge to update README.md. After that, remove changie files and folders related to current minor version. Commit.
  4. Bump version of dbt-tests-adapter. Commit.
  5. Merge these 2 commits into the master branch. Add a Skip Changlelog label to the PR.

Continue with the next steps for a minor version bump. Start from this point for a patch version bump:

  1. Run Version Bump workflow. The major and minor part of the dbt version are used to associate dbt-trino's version with the dbt version.
  2. Merge the bump PR. Make sure that test suite pass.
  3. Run dbt-trino release workflow to release dbt-trino to PyPi and GitHub.

Backport process

Sometimes it is necessary to backport some changes to some older versions. In that case, create branch from x.x.latest branch. There is a x.x.latest for each minor version, e.g. 1.3.latest. Make a fix and open PR back to x.x.latest. Create changelog by changie new as ususal, as separate changlog for each minor version is kept on every x.x.latest branch. After merging, to make a release of that version, just follow instructions from Release process section, but run every workflow on x.x.latest branch.

Code of Conduct

Everyone interacting in the dbt project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the PyPA Code of Conduct.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbt_trino-1.9.2.tar.gz (30.9 kB view details)

Uploaded Source

Built Distribution

dbt_trino-1.9.2-py3-none-any.whl (36.5 kB view details)

Uploaded Python 3

File details

Details for the file dbt_trino-1.9.2.tar.gz.

File metadata

  • Download URL: dbt_trino-1.9.2.tar.gz
  • Upload date:
  • Size: 30.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.3

File hashes

Hashes for dbt_trino-1.9.2.tar.gz
Algorithm Hash digest
SHA256 622c3ed1239ff39b48891e6c9208f549dd62635dc4569dad783d7e555292fb63
MD5 868cbdf8481b58b8bad25a46089abba2
BLAKE2b-256 dabaf9c6ca75d2c4a694f43ad4914437ec3054c617f73d98469ec43a16da4c0a

See more details on using hashes here.

File details

Details for the file dbt_trino-1.9.2-py3-none-any.whl.

File metadata

  • Download URL: dbt_trino-1.9.2-py3-none-any.whl
  • Upload date:
  • Size: 36.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.3

File hashes

Hashes for dbt_trino-1.9.2-py3-none-any.whl
Algorithm Hash digest
SHA256 0f73f2fe5304f882cb2e192d5de2712f3e66785a65d98b4fd41fbd238bdc848a
MD5 25380120fa42faeddd18a40b0413a2e8
BLAKE2b-256 3498915efba18ae582d0e17fe464954e6d4f1a52863183fe818e35b5ba6a8dbd

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page