Skip to main content

Data engineering & Data science Pipeline Framework

Project description

pygyver

Contains Data Enginner, Data Scientist and 3rd party integration tools capabilities.

Install

pygyver can be installed via pip

pip install pygyver

env var required

PROJECT_ROOT=  "root of he project"

AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=
AWS_S3_BUCKET=
AWS_S3_ROOT=

GOOGLE_APPLICATION_CREDENTIALS="path to the json token file"
BIGQUERY_PROJECT= "default name of the project in BigQuery"

etl

data_prep

Pypi release

Developing locally

source load-local-vars.sh --> sets local vars
make build --> build container locally
make run-tests --> executes the tests

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pygyver-0.1.0.4.tar.gz (40.5 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page