databricks-bundle 0.7.0a1

Databricks bundle for the Pyfony framework

Databricks bundle

This bundle allows you to write beautiful function-based notebooks.

Compared to bare notebooks, the function-based approach brings the following advantages:

• create and publish auto-generated documentation and lineage of notebooks and pipelines (Bricksflow PRO)
• write much cleaner notebooks with properly named code blocks
• (unit)test specific notebook functions with ease
• use YAML to configure your notebooks for given environment (dev/test/prod/...)
• utilize pre-configured objects to automate repetitive tasks

Function-based notebooks have been designed to provide the same user-experience as bare notebooks. Just write the function, annotate it with the @notebookFunction decorator and run the cell.

This bundle is the main part of the Bricksflow framework.

Installation

Install the bundle via Poetry:

$ poetry add databricks-bundle && poetry add databricks-connect --dev

Usage

1. Using pre-configured objects
2. Configuring notebook functions
3. Databricks Connect setup