Skip to main content

A common set of python libraries for DataBricks

Project description

atc-dataplatform

A common set of python libraries for DataBricks

Important Notes

This package can not be run or tested without access to pyspark. However, installing pyspark as part of our installer gave issues when other versions of pyspark were needed. Hence we took out the dependency from our installer.

Installation

Get it from PyPi PyPI version

pip install atc-dataplatform

Development Notes

To prepare for development please install these additional requirements:

  • Java 8
  • pip install -r test_requirements.txt

Then install the package locally

python setup.py develop

Git Hooks

Please install the git hooks in your repository by running

python utilities/git_hooks.py

The hooks will help you catch problems locally instead of having to wait for the PR pipeline to fail.

Testing

After installing the dev-requirements, execute tests by running

pytest tests

If these tests don't pass, PRs will not be accepted. If you add features, please include tests for these features.

General Project Info

Github top language Github stars Github forks Github size Issues Open

Packages

Build Status

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

atc-dataplatform-1.0.14.tar.gz (40.9 kB view hashes)

Uploaded Source

Built Distribution

atc_dataplatform-1.0.14-py3-none-any.whl (39.1 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page