Converting kedro pipelines to argo pipelines.
Project description
Converting kedro pipelines to argo pipelines.
Free software: BSD 3-Clause License
Installation
pip install kedro-argo
You can also install the in-development version with:
pip install https://github.com/nraw/kedro-argo/archive/master.zip
Requirements
To be used with Kedro, so it’s assumed this package is used with a Kedro project. An image name is needed. You can generate one with Kedro-docker. Argo CLI is needed for the deployment step. As the workflow will be in Argo, all datasets should be somehow passed between containers or else the pipeline will fail.
Usage
When installed, argo should be visible under your kedro commands
kedro
Get the kedro.yaml and the template file by running
kedro argo IMAGE_NAME
You can now run:
ytt -f templates > argo.yaml
or if you prefer in Docker:
docker run --rm -it --name ytt -v $(pwd)/templates:/templates gerritk/ytt:latest -f /templates > argo.yaml
and finally
argo submit --watch argo.yaml
Documentation
Development
To run the all tests run:
tox
Note, to combine the coverage data from all the tox environments run:
Windows |
set PYTEST_ADDOPTS=--cov-append tox |
---|---|
Other |
PYTEST_ADDOPTS=--cov-append tox |
Changelog
0.0.4 (2020-03-07)
Refactoring and initial adaptation
Inclusion of tests
0.0.0 (2020-03-07)
First release on PyPI.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for kedro_argo-0.0.6-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5548d4d5053a1d3aa980e1b8702832dae5d56538720c30448da4c933b5a3d551 |
|
MD5 | 13ad9aa33c760802967fdcc652f7f778 |
|
BLAKE2b-256 | 0a39e84ec783ab408c27fa9a14beedcae3d6dd3a4c723c05b4bd177982ef3597 |