lakehouse-ns 0.1.dev10

A library with the Lakehouse Framework

1. Get started

Requires to have installed one of the following:

• pyspark and delta-spark
• Databricks Connect
• Spark and Delta Connect

pip install lakehouse-ns

from lakehouse import bronze, silver

spark = <Your Spark Session>

spark.sql(f"CREATE SCHEMA IF NOT EXISTS <catalog>.<schema>")

options = {
    "catalog": "<catalog>",
    "target_schema": "<schema>" 
}


class StarWarsBronze(bronze.BronzeOverwrite):
    def load(self, table):
        return spark.read.format("SWAPI").load(table)
    
bronze_instance = StarWarsBronze(spark, **options)
bronze_instance.execute_one("people")

See samples here: https://github.com/datanikkthegreek/lakehouse-docu/tree/main/samples

2. Releases

• Bump up manually the release version in pyproject.toml

• pip install setuptools wheel twine

• python setup.py bdist_egg --exclude-source-files

• wheel convert dist/*.egg or

• python setup.py bdist_wheel

• twine upload dist/*.whl

• Enter the token