A library with the Lakehouse Framework
Project description
1. Get started
Requires to have installed one of the following:
- pyspark and delta-spark
- Databricks Connect
- Spark and Delta Connect
pip install lakehouse-ns
from lakehouse import bronze, silver
spark = <Your Spark Session>
spark.sql(f"CREATE SCHEMA IF NOT EXISTS <catalog>.<schema>")
options = {
"catalog": "<catalog>",
"target_schema": "<schema>"
}
class StarWarsBronze(bronze.BronzeOverwrite):
def load(self, table):
return spark.read.format("SWAPI").load(table)
bronze_instance = StarWarsBronze(spark, **options)
bronze_instance.execute_one("people")
See samples here: https://github.com/datanikkthegreek/lakehouse-docu/tree/main/samples
2. Releases
-
Bump up manually the release version in pyproject.toml
-
pip install setuptools wheel twine
-
python setup.py bdist_egg --exclude-source-files
-
wheel convert dist/*.egg
or -
python setup.py bdist_wheel
-
twine upload dist/*.whl
-
Enter the token
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
File details
Details for the file lakehouse_ns-0.1.dev10-py311-none-any.whl
.
File metadata
- Download URL: lakehouse_ns-0.1.dev10-py311-none-any.whl
- Upload date:
- Size: 21.8 kB
- Tags: Python 3.11
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.11.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a9bf8ddbf5ba79909f0cf5191f6d1c4dfd46021601220fd741cc1f0aca7dac1e |
|
MD5 | 561bc2d26184e479bec10884ee92c3b0 |
|
BLAKE2b-256 | 14e1cc8556830bf61d205e941517cacab01882dc06deadcba04c5fce10f37776 |