CLI tool for datalake operation

Project description

shopcloud-datalake-cli

Datalake ETL tool

mkdir config-dir
$ datalake --project="evident-castle-411608" --base-dir="examples" config create
$ datalake --project="evident-castle-411608" --base-dir="examples" config sync
$ datalake --project="evident-castle-411608" --base-dir="examples" run --partition-date=2024-01-19
$ datalake --project="evident-castle-411608" --base-dir="examples" run <table> --partition-date=2024-01-19

Development

# run unit tests
$ python3 -m unittest
# run unit tests with coverage
$ python3 -m coverage run --source=tests,shopcloud_datalake -m unittest discover && python3 -m coverage html -d coverage_report
$ python3 -m coverage run --source=tests,shopcloud_datalake -m unittest discover && python3 -m coverage xml

Project details

Release history Release notifications | RSS feed

1.5.0

Mar 26, 2024

1.4.0

Feb 10, 2024

1.3.0

Feb 6, 2024

This version

1.2.0

Jan 22, 2024

1.1.0

Jan 21, 2024

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

shopcloud-datalake-1.2.0.tar.gz (17.0 kB view hashes)

Uploaded Jan 22, 2024 Source

Built Distribution

shopcloud_datalake-1.2.0-py3-none-any.whl (23.5 kB view hashes)

Uploaded Jan 22, 2024 Python 3

Hashes for shopcloud-datalake-1.2.0.tar.gz

Hashes for shopcloud-datalake-1.2.0.tar.gz
Algorithm	Hash digest
SHA256	`dcd39e3b5e33807c8a7a3eb516f4247a43923f689ce9af4d4bd92a774b09fa6d`
MD5	`b2a6b89c6d495e471cfbc1f94935d606`
BLAKE2b-256	`0e313cab8ad2b655f45d693d630757f1789786346a51fe173c21580b802771af`

Hashes for shopcloud_datalake-1.2.0-py3-none-any.whl

Hashes for shopcloud_datalake-1.2.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`322868752e48b4e3a54c6461fa6576879143c5a6f08cc12e135dd90f45831c32`
MD5	`e4df959096576cc308b129931578448c`
BLAKE2b-256	`ffa55ff861d2248499df7ce365ea1404e33f877e9d06694bc6c5b9ae97775236`