mlflow-ray-serve 0.2.0

Ray Serve MLflow deployment plugin

MLflow-Ray-Serve

An experimental plugin that integrates Ray Serve with the MLflow pipeline. mlflow-ray-serve enables MLflow users to deploy MLflow models at scale on Ray Serve.

This plugin implements the Python API and command-line interface for MLflow deployment plugins.

Installation

pip install mlflow-ray-serve

The following packages are required and will be installed along with the plugin:

1. "ray[serve]"
2. "mlflow>=1.12.0"

This plugin requires Ray version 1.7.0 or greater.

Usage

This plugin must be used with a detached Ray Serve instance running on a Ray cluster. An easy way to set this up is by running the following two commands:

ray start --head # Start a single-node Ray cluster locally.
serve start # Start a detached Ray Serve instance.

The API is summarized below. For full details see the MLflow deployment plugin Python API and command-line interface documentation.

Create deployment

Deploy a model built with MLflow using Ray Serve with the desired configuration parameters; for example, num_replicas. Currently this plugin only supports the python_function flavor of MLflow models, and this is the default flavor.

CLI

mlflow deployments create -t ray-serve -m <model uri> --name <deployment name> -C num_replicas=<number of replicas>

Python API

from mlflow.deployments import get_deploy_client
target_uri = 'ray-serve'
plugin = get_deploy_client(target_uri)
plugin.create_deployment(
    name=<deployment name>,
    model_uri=<model uri>,
    config={"num_replicas": 4})

Update deployment

Modify the configuration of a deployed model and/or replace the deployment with a new model URI.

CLI

mlflow deployments update -t ray-serve --name <deployment name> -C num_replicas=<new number of replicas>

Python API

plugin.update_deployment(name=<deployment name>, config={"num_replicas": <new number of replicas>})

Delete deployment

Delete an existing deployment.

CLI

mlflow deployments delete -t ray-serve --name <deployment name>

Python API

plugin.delete_deployment(name=<deployment name>)

List deployments

List the names of all the models deployed on Ray Serve. Includes models not deployed via this plugin.

CLI

mlflow deployments list -t ray-serve

Python API

plugin.list_deployments()

Get deployment details

CLI

mlflow deployments get -t ray-serve --name <deployment name>

Python API

plugin.get_deployment(name=<deployment name>)

Run prediction on deployed model

For the prediction inputs, DataFrame, Tensor and JSON formats are supported by the Python API. To invoke via the command line, pass in the path to a JSON file containing the input.

CLI

mlflow deployments predict -t ray-serve --name <deployment name> --input-path <input file path> --output-path <output file path>

output-path is an optional parameter. Without it, the result will be printed in the terminal.

Python API

plugin.predict(name=<deployment name>, df=<prediction input>)

Plugin help

Prints the plugin help string.

CLI

mlflow deployments help -t ray-serve