databricks-bundle 0.8.0a1

Databricks bundle for the Daipe framework

Databricks bundle

This bundle allows you to write beautiful function-based notebooks.

Compared to bare notebooks, the function-based approach brings the following advantages:

1. create and publish auto-generated documentation and lineage of notebooks and pipelines (Daipe PRO)
2. write much cleaner notebooks with properly named code blocks
3. (unit)test specific notebook functions with ease
4. use YAML to configure your notebooks for given environment (dev/test/prod/...)
5. utilize pre-configured objects to automate repetitive tasks

Function-based notebooks have been designed to provide the same user-experience as bare notebooks. Just write the function, annotate it with the @notebookFunction decorator and run the cell.

This bundle is the main part of the Daipe framework.

Installation

Install the bundle via Poetry:

$ poetry add databricks-bundle && poetry add databricks-connect --dev

Usage

1. Writing function-based notebooks
2. Using pre-configured objects
3. Configuring notebook functions
4. Databricks Connect setup