dadude

Lakehouse management and orchestration tool.

Status

The ultimate target lakehouse is the unity catalog, which is to provide benefits from both deltalake and iceberg.
To support writing data to the lakehouse, we currently use the deltalake-rs python bindings.
For testing purposes, we use the minio instance at http://192.168.18.206:9000.

Installation

This package is published to the PyPI repository, so you can install it using pip anywhere:

pip install dadude

Usage

First export the following environment variables:

export STORAGE_ACCESS_KEY_ID=xxx STORAGE_SECRET_ACCESS_KEY=xxx STORAGE_ENDPOINT_URL=http://192.168.18.206:9000

Then you can use the dadude module CLI to interact with the lakehouse:

python -m dadude.cli.read_table staging <table_name> --save_dir <save_dir>
python -m dadude.cli.write_table write_json_table --local_json_file_path data/silver/xxx.json

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dadude-0.3.5.tar.gz (141.0 kB view details)

Uploaded Jan 16, 2025 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

dadude-0.3.5-py3-none-any.whl (21.8 kB view details)

Uploaded Jan 16, 2025 Python 3

File details

Details for the file dadude-0.3.5.tar.gz.

File metadata

Download URL: dadude-0.3.5.tar.gz
Upload date: Jan 16, 2025
Size: 141.0 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.5.1

File hashes

Hashes for dadude-0.3.5.tar.gz
Algorithm	Hash digest
SHA256	`17c4134650f845287a85866530c2b762f996392d2238ac34ac5e02af27f94ba8`
MD5	`6052000c0a876594af717213c758ab82`
BLAKE2b-256	`9d9ef488286a00a3d93351eabcbb01055709f1851a9f8380a9a80b815a50755a`

See more details on using hashes here.

File details

Details for the file dadude-0.3.5-py3-none-any.whl.

File metadata

Download URL: dadude-0.3.5-py3-none-any.whl
Upload date: Jan 16, 2025
Size: 21.8 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.5.1

File hashes

Hashes for dadude-0.3.5-py3-none-any.whl
Algorithm	Hash digest
SHA256	`ebf6f4c0968fb1e3a736b3bfa995a8ba32e939460605b7f4b93c93dcf2d01772`
MD5	`0ea85a964b609658298e0d3d255c2d38`
BLAKE2b-256	`d069125f9c6138054363123041a1092114e25e2ac8556ceeee0a2709fe76d7d2`