Skip to main content

Replicate humanity data in databricks

Project description

humanity_etl

Replicate data from humanity in databricks

pypi

Usage

CLI

python -m humanity_etl --table <table-name>

where table-name is one of the following: timeclocks or ALL.

API

from humanity_etl.tables.timeclocks import timeclocks

timeclocks()

Further configs are set via environment variables:

Further configs are set via environment variables:

  • *_ACCESS_TOKEN: API access
  • APP_ENV: <development/production> - debug mode
  • SAMPLE_OUTPUT: <True/False> - Save dataframes to csv

DEV

Create venv

python -m venv env

Activate venv

  • unix
source env/bin/activate
  • windows
env\Scripts\activate.bat

Install Packages

pip install -r requirements.txt

Test

make test

Format

make format
make lint

Version & Release

bump2version <major/minor/patch>
make release

note Don't forget to git push with --tags

pre-commit

Setup

pre-commit install

Run all

make pre-commit

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

humanity_etl-0.1.1.tar.gz (7.6 kB view details)

Uploaded Source

File details

Details for the file humanity_etl-0.1.1.tar.gz.

File metadata

  • Download URL: humanity_etl-0.1.1.tar.gz
  • Upload date:
  • Size: 7.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.14.0 pkginfo/1.9.6 requests/2.28.2 setuptools/58.1.0 requests-toolbelt/0.10.1 tqdm/4.65.0 CPython/3.9.16

File hashes

Hashes for humanity_etl-0.1.1.tar.gz
Algorithm Hash digest
SHA256 29f684add1063687cd40a14d5c85120146100950702691e95ff15c90ecd1f5ec
MD5 3d974a683fdbc469fc0db4cdfc5b204a
BLAKE2b-256 3f0b884b9d8d80cfb22aa67e34bb41d7538673e538c577d01ed99ba3fc638dd9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page