datalake-bundle 0.5.0

DataLake tables management bundle for the Bricksflow Framework

Datalake bundle

This bundle provides everything you need to create and manage a Databricks-based DataLake(House):

• Tools to simplify & automate table creation, updates and migrations.
• Explicit table schema enforcing for Hive tables, CSVs, ...
• Decorators to write well-maintainable and self-documented function-based notebooks
• Rich configuration options to customize naming standards, paths, and basically anything to match your needs

Installation

Install the bundle via Poetry:

$ poetry add datalake-bundle

Usage

1. Recommended notebooks structure
2. Defining DataLake tables
3. Using datalake-specific notebook functions
4. Using table-specific configuration
5. Tables management
6. Parsing fields from table identifier