Skip to main content

Converting kedro pipelines to argo pipelines.

Project description

Converting kedro pipelines to argo pipelines.

  • Free software: BSD 3-Clause License


pip install kedro-argo

You can also install the in-development version with:

pip install


  • To be used with Kedro, so it’s assumed this package is used with a Kedro project.

  • Argo CLI is needed for the deployment step. It’s also assumed that Argo is already installed on your kuberentes instance.

  • You must specify an image name as a parameter. You can generate the image using Kedro-docker.

  • As the workflow will be in Argo, which means every step will run in its own container. Therefore, all datasets should be somehow passed between containers or else the pipeline will fail. This means either all datasets should be saved externally (S3, Azure, etc.) or in a shared folder that your deployment would have access to.


When installed, argo should be visible under your kedro commands, if you’re in a kedro project


Then you have two options for obtaining the yaml file, namely via Helm or via ytt.


kedro argo IMAGE_NAME

Add this repository to your helm charts:

helm repo add kedro-argo

Then either directly install it by passing the kedro.yaml for input values

helm install -f templates/kedro.yaml kedro-argo kedro-argo/kedro-argo

Or clone it to your repository and change anything that you would still need:

helm pull kedro-argo/kedro-argo --untar


Get the kedro.yaml file by running

kedro argo --ytt IMAGE_NAME

You can now run:

ytt -f templates > argo.yaml

or if you prefer in Docker:

docker run --rm -it --name ytt -v $(pwd)/templates:/templates gerritk/ytt:latest -f /templates > argo.yaml

and finally

argo submit --watch argo.yaml



To run the all tests run:


Note, to combine the coverage data from all the tox environments run:


set PYTEST_ADDOPTS=--cov-append


PYTEST_ADDOPTS=--cov-append tox


0.0.8 (2020-04-27)

  • Changed the ytt option to be a flag

0.0.7 (2020-03-27)

  • Changed the default templating option to be Helm instead of ytt

0.0.5 (2020-03-08)

  • Dirty names are now transformed to make less likely that symbols break Argo

0.0.4 (2020-03-07)

  • Refactoring and initial adaptation

  • Inclusion of tests

0.0.0 (2020-03-07)

  • First release on PyPI.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kedro-argo-0.0.9.tar.gz (264.0 kB view hashes)

Uploaded Source

Built Distribution

kedro_argo-0.0.9-py2.py3-none-any.whl (6.8 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page