Data engineering & Data science Pipeline Framework
Project description
pygyver
Contains Data Enginner, Data Scientist and 3rd party integration tools capabilities.
Install
pygyver can be installed via pip
pip install pygyver
env var required
PROJECT_ROOT= "root of he project"
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=
AWS_S3_BUCKET=
AWS_S3_ROOT=
GOOGLE_APPLICATION_CREDENTIALS="path to the json token file"
BIGQUERY_PROJECT= "default name of the project in BigQuery"
etl
data_prep
Pypi release
- create a PR
- ask for code review
- merge to master branch
- create a new Release in https://github.com/madedotcom/pygyver/releases
Developing locally
source load-local-vars.sh --> sets local vars
make build --> build container locally
make run-tests --> executes the tests
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pygyver-0.1.0.7.tar.gz
(42.0 kB
view details)
File details
Details for the file pygyver-0.1.0.7.tar.gz.
File metadata
- Download URL: pygyver-0.1.0.7.tar.gz
- Upload date:
- Size: 42.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.3.1 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.7.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8b51b76ceba325868c60b49aeb93f6b65f731676ee0ac3f4919a3f3c6f396a55
|
|
| MD5 |
6191c69b69f4e4429949f77ba7d940e7
|
|
| BLAKE2b-256 |
15f1606ce58c6601ffe1accf12ae64f2ac832a214539e73dbfe57091ad4ba494
|
