A Python Client for Bakplane
Project description
Bakplane Client for Python
Bakplane is the orchestration backbone for Dominus, FinFlo, and Fabrik.
This SDK can be used to control ingestion sessions, mastering executions, plugin installation, etc.
As an example, we use this package to with Airflow to control ingestion orchestration.
Installation
Installing the bakplane python client takes just a few seconds:
pip install bakplane-python-sdk
Bindings
If you're a contributor and would like to generate the bindings from proto then run:
scripts/proto.sh
Examples
If you want to explore the notebooks then install jupyter
:
pip install jupyter
cd bakplane/examples
jupyter notebook
Packages
If you want to build a new bakplane binary then run:
git tag -a v1.1.1 -m "Bakplane SDK Release"
git push origin v1.1.1
Then you may find the packages available here: https://github.com/openaristos/bakplane-python-sdk/releases
Spark
Start spark in Docker:
docker run -it --rm -p 8888:8888 --name pyspark jupyter/pyspark-notebook
Copy the latest python package, and the notebooks, onto the cluster:
scripts/package.sh
docker cp dist/bakplane-python-sdk-1.1.1.tar.gz pyspark:/tmp/
docker cp bakplane/examples/notebooks/spark.ipynb pyspark:/home/jovyan/work/
Note: use host.docker.internal
instead of localhost
when testing.
Then start the jupyter notebook by clicking on the link provided by the Docker container.
Documentation
If you're interested in learning more then read the documentation: https://oa.docs.openaristos.io/
Copyright (C) 2020 Aristos Data, LLC
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for bakplane-python-sdk-1.1.15.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | db043933be995e3b8f2bf457eb971310b0fc4269f1137c81d62c89eeba4a3b91 |
|
MD5 | 8d1aacae75a23ae7d95800218bc66f31 |
|
BLAKE2b-256 | 22a20b55392af76e60c1ab015abc8da48b61c520969610004bb9bd15e9099a26 |
Hashes for bakplane_python_sdk-1.1.15-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b44261652969cf88580e5d87ac411ad1e2dc3cfcd0c053380eec68a1d355d6f |
|
MD5 | de415c79d1a2cfeeb85c10f0c007249d |
|
BLAKE2b-256 | ba06e46fe3703704c49a60d9e2a20b2b102d43f67463f39000dd08af7ecfac8a |