atc-dataplatform 1.0.50

A common set of python libraries for DataBricks

atc-dataplatform

A common set of python libraries for DataBricks.

Important Notes

This package can not be run or tested without access to pyspark. However, installing pyspark as part of our installer gave issues when other versions of pyspark were needed. Hence we took out the dependency from our installer.

Installation

Get it from PyPi

pip install atc-dataplatform

Development Notes

To prepare for development please install these additional requirements:

• Java 8
• pip install -r test_requirements.txt

Then install the package locally

python setup.py develop

Git Hooks

Please install the git hooks in your repository by running

python utilities/git_hooks.py

The hooks will help you catch problems locally instead of having to wait for the PR pipeline to fail.

Testing

After installing the dev-requirements, execute tests by running

pytest tests

If these tests don't pass, PRs will not be accepted. If you add features, please include tests for these features.

General Project Info

Packages

Build Status