Data engineering & Data science Framework
Contains Data Enginner, Data Scientist and 3rd party integration tools capabilities
py-test-utility can be installed via pip
pip install ??find the name??
To access BigQuery you will need following environment variables:
BIGQUERY_PROJECT- name of the project in BigQuery. Default project cannot be changed in the code.
BIGQUERY_ACCESS_TOKEN_PATH- path to the json token file.
BigQuery functions wrap
bigrquery functions to provide higher level API removing boilerplate instructions of the lower level API.
Contains methods to extract the equivalent json from csv with nested and repeated records structures
- if successfuls return the json obj extracted from the csv
- return 0 if successfuls
- create new line delimiter "output_file_name" file
Generating distribution archives
python3 -m pip install --user --upgrade setuptools wheel
python3 -m pip install --user --upgrade twine
Now run this command from the same directory where setup.py is located:
python3 setup.py sdist bdist_wheel
Uploading the distribution archives
python3 -m twine upload --repository-url dist/*
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.