Skip to main content

A common set of python libraries for DataBricks

Project description

atc-dataplatform

A common set of python libraries for DataBricks.

Important Notes

This package can not be run or tested without access to pyspark. However, installing pyspark as part of our installer gave issues when other versions of pyspark were needed. Hence we took out the dependency from our installer.

Installation

Get it from PyPi PyPI version PyPI

pip install atc-dataplatform

Development Notes

To prepare for development please install these additional requirements:

  • Java 8
  • pip install -r test_requirements.txt

Then install the package locally

python setup.py develop

Git Hooks

Please install the git hooks in your repository by running

python utilities/git_hooks.py

The hooks will help you catch problems locally instead of having to wait for the PR pipeline to fail.

Testing

After installing the dev-requirements, execute tests by running

pytest tests

If these tests don't pass, PRs will not be accepted. If you add features, please include tests for these features.

General Project Info

Github top language Github stars Github forks Github size Issues Open

Packages

Build Status

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

atc-dataplatform-1.0.50.tar.gz (65.4 kB view hashes)

Uploaded source

Built Distribution

atc_dataplatform-1.0.50-py3-none-any.whl (63.1 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page