klops 0.0.4

Klops: Koin Machine Learning Ops

Klops: Koin MLOps

Build your end-to-end machine learning projects within one tool.

Getting Started • Documentation • Support • Contribution

Klops is an End-to-End machine learning development pipeline ops. Its build on top of Seldon Core, MLflow and DVC. The goal of this project is to make easier for Data Scientist to develop, maintain, log their experiments and deploy their machine learning projects.

Prerequisites

• Seldon Core Installed on your Kubernetes cluster. Guidance here
• MLflow Tracking Server deployed. Recommended to use the schenario 5 here
• Local or remote storage such as Amazon S3 or Google Cloud Storage (GCS) or similar.

Installation

From PyPI

Using pip is the best option if you want to get the stable version and easier way to install.

$ pip install klops

Build From Source

This is the best way to get the lastest published library, including the experimental version.

Clone this repo

$ git clone https://gitlab-engineering.koinworks.com/data-team/klops.git

Change your directory to your klops folder:

$ cd klops

Install

$ python setup.py install

Basic Usages

Klops consists of three modules. Versioning, Experiment and Deployment.

Klops Experiment

Klops Experiment is a class that wraps the MLflow Tracking. Below are the simple example to begin with.

from klops.experiment import Experiment
from sklearn.datasets import load_iris
from sklearn.naive_bayes import GaussianNB

experiment = Experiment(name="your-experiment-name", tracking_uri="http://<your-mlflow-host>:<port>")

X, y = load_iris(return_X_y=True)
HYPERPARAMETERS = {
    "n_estimators": 20,
    "max_depth": 10,
    "min_samples_split": 5
}

experiment.start(GaussianNB, x_train_data=X, y_train_data=y, tuner_args=HYPERPARAMETERS)

You would see your experiment result in your Mlflow Tracking UI like below:
For the complete tutorials could be seen through this documentation.

Klops Versioning

Klops Versioning is a kind of version control based on DVC. This module wrapped the commandline and python APIs from DVC.

from klops.versioning import Versioning

versioning = Versioning()

# Track your file into dvc
versioning.add("myfile.csv")

# Add your DVC repository storage
versioning.add_remote("gs://your-bucket-name/your-path/")

# Push your changes to DVC
versioning.push()

Complete examples could be seen on this tutorial page.

Klops Deployment

Klops Deployment is a module to deploy the development machine learning projects into Seldon Core instance. Below is the example on how to done with.

from klops.seldon_core import SeldonDeployment
from klops.seldon_core.auth import GKEAuthentication

gke = GKEAuthentication(
    project_id="your-project-id",
    zone="your-project-region",
    cluster_id="your-cluster-id")
deployment = SeldonDeployment(gke, "seldon")

## Config file could be json or yaml/yml file.
config = deployment.load_deployment_configuration("<deployment-config>.json")
deployment.deploy(config)

Now you can access your API through this doc http://<ingress_url>/seldon/<namespace>/<model-name>/api/v1.0/doc/. Example result:

Development

We are open for anyone who wants to contribute! Please read the CONTRIBUTING.md guide.