kedro-argo 0.0.5

Converting kedro pipelines to argo pipelines.

Converting kedro pipelines to argo pipelines.

• Free software: BSD 3-Clause License

Installation

pip install kedro-argo

You can also install the in-development version with:

pip install https://github.com/nraw/kedro-argo/archive/master.zip

Requirements

To be used with Kedro, so it’s assumed this package is used with a Kedro project. An image name is needed. You can generate one with Kedro-docker. Argo CLI is needed for the deployment step. As the workflow will be in Argo, all datasets should be somehow passed between containers or else the pipeline will fail.

Usage

When installed, argo should be visible under your kedro commands

kedro

Get the kedro.yaml and the template file by running

kedro argo IMAGE_NAME

You can now run:

ytt -f templates > argo.yaml

or if you prefer in Docker:

docker run --rm -it --name ytt -v $(pwd)/templates:/templates gerritk/ytt:latest -f /templates > argo.yaml

and finally

argo submit --watch argo.yaml

Documentation

https://kedro-argo.readthedocs.io/

Development

To run the all tests run:

tox

Note, to combine the coverage data from all the tox environments run:

Windows	set PYTEST_ADDOPTS=--cov-append tox
Other	PYTEST_ADDOPTS=--cov-append tox

Changelog

0.0.4 (2020-03-07)

• Refactoring and initial adaptation

• Inclusion of tests

0.0.0 (2020-03-07)

• First release on PyPI.