dbt-databricks-runner 0.1.1

Overview

This repository contains the dbt-runner to run dbt-spark on Databricks through:

• VSCode extension for local development
• Deployed with Databricks Job Clusters using spark session.

Prerequisites

Before setting up, ensure you have the following tools installed:

• Databricks CLI: Installation Guide
• VSCode Databricks Extension: Available via VSCode marketplace.
• uv: Installation Guide

Set-Up Instructions

1. Install this repository with pip/uv

2. Configure Databricks CLI:

   • Run databricks configure --token and follow prompts to set up authentication.

3. Set up DBT profiles:

   • Create or update your ~/.dbt/profiles.yml with your Databricks connection details.

4. Run feature preparation pipelines:

   • Use uv run dbt run to execute pipelines.
   • Use uv run dbt test to validate models.