Starters for kedro projects to simplify pipelines deployment using GetInData plugins
Project description
Kedro starters by GetInData
In GetInData we deploy Kedro-based projects to different environments (on-premise and cloud). This repository hosts the starters with a few deployment recipes, including the ones that use our plugins.
Available starters
- pyspark-iris-running-on-gke uses Google Kubernetes Engine to run Spark-powered kedro pipeline in a distributed manner.
- pyspark-iris-running-on-gcp-dataproc-serverless uses Google Cloud Dataproc Batches to run Spark-powered kedro pipeline in a distributed manner on Severless Spark.
Starters development
- Clone the repository and switch to
develop - Run
poetry install - Run
source $(poetry env info --path)/bin/activateNote: when you useconda, you need the extra step ofconda deactivateto avoid conflict between thecondaandvenv - Install kedro
pip install kedro==0.18.3 - Run
kedro new -s <name-of-the-starter>
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
getindata_kedro_starters-0.2.0.tar.gz
(161.0 kB
view hashes)
Built Distribution
Close
Hashes for getindata_kedro_starters-0.2.0.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 239ecdbc11fdac33037c49ba725e5d23711ecfdbbc1dc16a7829e2e0494ef439 |
|
| MD5 | 9c8f0bfa0671e76a3889ec02c08415d4 |
|
| BLAKE2b-256 | 3aced8e6c1604cbe15d5ffe8b736f6d4b5e05d6e84e484d699f4f1826a7cdd79 |
Close
Hashes for getindata_kedro_starters-0.2.0-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 7e7fb1293aa0ea6a98e1173e8594280fd13937f042d268a588d55576a4b9d833 |
|
| MD5 | 444abff7ed0d1f3d947fbcc2dc33af4c |
|
| BLAKE2b-256 | 2878e1665a9b2a1c320bfffb5a4f661a701c3b29bccbcf6edf32f9f67e5d35dc |
