kfp 1.8.22

KubeFlow Pipelines SDK

kfp: Kubeflow Pipelines SDK

Kubeflow Pipelines is a platform for building and deploying portable, scalable machine learning workflows based on Docker containers within the Kubeflow project.

Use Kubeflow Pipelines to compose a multi-step workflow (pipeline) as a graph of containerized tasks using Python code and/or YAML. Then, run your pipeline with specified pipeline arguments, rerun your pipeline with new arguments or data, schedule your pipeline to run on a recurring basis, organize your runs into experiments, save machine learning artifacts to compliant artifact registries, and visualize it all through the Kubeflow Dashboard.

Documentation

• Kubeflow Pipelines Overview
• SDK Overview
• SDK API Documentation

Installation

To install the latest stable release, run:

pip install kfp

Getting started

The following is an example of a simple pipeline with one Python function-based component used in two separate tasks to do basic addition:

import kfp
from kfp.components import create_component_from_func
import kfp.dsl as dsl

def add(a: float, b: float) -> float:
    '''Calculates sum of two arguments'''
    return a + b


# create a component using the add function
add_op = create_component_from_func(add)

# compose your pipeline using the dsl.pipeline decorator
@dsl.pipeline(
    name='Addition pipeline',
    description='An example pipeline that performs addition calculations.')
def add_pipeline(
    a: float=1.0,
    b: float=7.0,
):
    first_add_task = add_op(a=a, b=4.0)
    second_add_task = add_op(a=first_add_task.output, b=b)

# instantiate a client and submit your pipeline with arguments
client = kfp.Client(host='<my-host-url>')
client.create_run_from_pipeline_func(
    add_pipeline, arguments={
        'a': 7.0,
        'b': 8.0
    })

For more information, refer to Building Python function-based components.