Combine duckdb-dbt and Kedro Dataset to easily read Kedro Dataset configs (yaml), enabling conversion of Kedro projects to dbt.
Project description
Combine duckdb-dbt and Kedro Datases to enable:
- extension of dbt to ingest wide array of data, and;
- conversion of Kedro projects to dbt by easily reading your Kedro data catalog configs (yaml files)
Demo
You can add your existing Kedro definitions to your dbt sources like so:
pip install dbt_duckdb_kedro_datasets
version: 2
sources:
- name: my_source # can call this anything
schema: main
meta:
plugin: dbt_duckdb_kedro_datasets # this library
tables:
- name: my_table # can call this anything
description: "A dbt_duckdb_kedro_datasets test"
meta:
type: pandas.CSVDataset
filepath: ./data/1_raw/bikes.csv # file to ingest
load_args:
sep: ','
Now we can access this CSV in dbt
select *
from {{ source('my_source', 'my_table') }}
For a more complete example look at this
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for dbt_duckdb_kedro_datasets-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 654dafc94feca8402e29506dbaae1db6a15727369ce7a4f248dded3602010eae |
|
MD5 | 03a513cfa81e7de7e091ae23ab2c7cf8 |
|
BLAKE2b-256 | 23ef33a064806071ab5b0e8c5e24fcbfa7d12c3c3725a10322b45c0d7644f249 |
Close
Hashes for dbt_duckdb_kedro_datasets-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d69c37c27d60f277e6ef9dd563af23e7b8c84a2f3aa4a656e4b6b37626aa61b |
|
MD5 | 2ab1768657ced352ac5f88818b393eb2 |
|
BLAKE2b-256 | 6f2ac3af71e864b220e85c631dee44b27974809299e4b7a33dc8bb8e8923fd3a |