Starters for kedro projects to simplify pipelines deployment using GetInData plugins
Project description
Kedro starters by GetInData
In GetInData we deploy Kedro-based projects to different environments (on-premise and cloud). This repository hosts the starters with a few deployment recipes, including the ones that use our plugins.
Available starters
- pyspark-iris-running-on-gke uses Google Kubernetes Engine to run Spark-powered kedro pipeline in a distributed manner.
- pyspark-iris-running-on-gcp-dataproc-serverless uses Google Cloud Dataproc Batches to run Spark-powered kedro pipeline in a distributed manner on Severless Spark.
Starters development
- Clone the repository and switch to
develop
- Run
poetry install
- Run
source $(poetry env info --path)/bin/activate
Note: when you useconda
, you need the extra step ofconda deactivate
to avoid conflict between theconda
andvenv
- Install kedro
pip install kedro==0.18.3
- Run
kedro new -s <name-of-the-starter>
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file getindata_kedro_starters-0.2.0.tar.gz
.
File metadata
- Download URL: getindata_kedro_starters-0.2.0.tar.gz
- Upload date:
- Size: 161.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.16
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 239ecdbc11fdac33037c49ba725e5d23711ecfdbbc1dc16a7829e2e0494ef439 |
|
MD5 | 9c8f0bfa0671e76a3889ec02c08415d4 |
|
BLAKE2b-256 | 3aced8e6c1604cbe15d5ffe8b736f6d4b5e05d6e84e484d699f4f1826a7cdd79 |
File details
Details for the file getindata_kedro_starters-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: getindata_kedro_starters-0.2.0-py3-none-any.whl
- Upload date:
- Size: 196.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.16
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e7fb1293aa0ea6a98e1173e8594280fd13937f042d268a588d55576a4b9d833 |
|
MD5 | 444abff7ed0d1f3d947fbcc2dc33af4c |
|
BLAKE2b-256 | 2878e1665a9b2a1c320bfffb5a4f661a701c3b29bccbcf6edf32f9f67e5d35dc |