The buenavista adapter plugin for dbt (data build tool)
Project description
dbt-buenavista
Buena Vista is a Postgres proxy server written in Python. It is intended to serve as the Controller in the Model-View-Controller architecture for data applications, providing a locus of control for data teams in between the data warehouse (model layer) and downstream data consumers like BI tools and notebooks (view layer.)
dbt is the best way to manage a collection of data transformations written in SQL or Python for analytics and data science.
The purpose of dbt-buenavista
is to explore some new techniques for creating dbt models that a proxy service like
Buena Vista makes possible, starting with a way of executing dbt's Python models inspired by the approach taken
by dbt-duckdb: instead of pushing the Python compute into the Python execution engines
supported by Snowflake, Databricks, and BigQuery, the dbt-buenavista adapter executes the Python models on the Buena Vista
server itself, reading in any data tables that are used in the computation as data frames (either Pandas or another data frame
library supported by the data warehouse) and writing the data frame back to the data warehouse when the computation is finished.
This approach makes it possible to support Python-based models against data warehouses that do not ship with a Python execution
engine (most notably Postgres and Redshift) and may well simplify the development and debugging workflows for teams that want to
include Python models in their data pipelines (e.g., print
and logging
statements will run on the Buena Vista server and can
be sent wherever they need to go to aid in debugging.)
Installation
Configuring Your Profile
The profile settings for a dbt-buenavista connection are nearly identical to the settings for a
dbt-postgres profile, since every
dbt-buenavista
adapter is a subclass of the dbt-postgres
adapter. Instead of setting the profile's type
field as
postgres
, you set it to one of the Buena Vista supported types, which are currently bv_duckdb
(for a Buena Vista server
powered by DuckDB) or bv_postgres
(for a Buena Vista server that is proxying a Postgres database.) The host
and port
settings are required and should be configured to point at the Buena Vista server that you are using.
default:
outputs:
dev:
type: <bv_duckdb|bv_postgres>
host: <host>
port: <port>
dbname: <dbname>
...
target: dev
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dbt_buenavista-1.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d1303a053e71b3dc26d1f85aa045658a58a95be2b76952314774afd86f2c1214 |
|
MD5 | 34972509dd34893b009c8a4cc058ff04 |
|
BLAKE2b-256 | 66bf25bc8b6ad6fa2b59de5888f3e013af2ce891e8e255359678d81b287b601c |