Skip to main content

Converting kedro pipelines to argo pipelines.

Project description

Converting kedro pipelines to argo pipelines.

  • Free software: BSD 3-Clause License

Installation

pip install kedro-argo

You can also install the in-development version with:

pip install https://github.com/nraw/kedro-argo/archive/master.zip

Requirements

To be used with Kedro, so it’s assumed this package is used with a Kedro project. An image name is needed. You can generate one with Kedro-docker. Argo CLI is needed for the deployment step. As the workflow will be in Argo, all datasets should be somehow passed between containers or else the pipeline will fail.

Usage

When installed, argo should be visible under your kedro commands

kedro

Get the kedro.yaml and the template file by running

kedro argo IMAGE_NAME

You can now run:

ytt -f templates > argo.yaml

or if you prefer in Docker:

docker run --rm -it --name ytt -v $(pwd)/templates:/templates gerritk/ytt:latest -f /templates > argo.yaml

and finally

argo submit --watch argo.yaml

Development

To run the all tests run:

tox

Note, to combine the coverage data from all the tox environments run:

Windows
set PYTEST_ADDOPTS=--cov-append
tox
Other
PYTEST_ADDOPTS=--cov-append tox

Changelog

0.0.4 (2020-03-07)

  • Refactoring and initial adaptation
  • Inclusion of tests

0.0.0 (2020-03-07)

  • First release on PyPI.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for kedro-argo, version 0.0.7
Filename, size File type Python version Upload date Hashes
Filename, size kedro_argo-0.0.7-py2.py3-none-any.whl (6.3 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size kedro-argo-0.0.7.tar.gz (16.2 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page