Skip to main content

Ray Serve MLflow deployment plugin

Project description


An experimental plugin that integrates Ray Serve with the MLflow pipeline. mlflow-ray-serve enables MLflow users to deploy MLflow models at scale on Ray Serve.

This plugin implements the Python API and command-line interface for MLflow deployment plugins.


pip install mlflow-ray-serve

The following packages are required and will be installed along with the plugin:

  1. "ray[serve]"
  2. "mlflow>=1.12.0"

This plugin requires Ray version 1.7.0 or greater.


This plugin must be used with a detached Ray Serve instance running on a Ray cluster. An easy way to set this up is by running the following two commands:

ray start --head # Start a single-node Ray cluster locally.
serve start # Start a detached Ray Serve instance.

The API is summarized below. For full details see the MLflow deployment plugin Python API and command-line interface documentation.

Create deployment

Deploy a model built with MLflow using Ray Serve with the desired configuration parameters; for example, num_replicas. Currently this plugin only supports the python_function flavor of MLflow models, and this is the default flavor.

mlflow deployments create -t ray-serve -m <model uri> --name <deployment name> -C num_replicas=<number of replicas>
Python API
from mlflow.deployments import get_deploy_client
target_uri = 'ray-serve'
plugin = get_deploy_client(target_uri)
    name=<deployment name>,
    model_uri=<model uri>,
    config={"num_replicas": 4})

Update deployment

Modify the configuration of a deployed model and/or replace the deployment with a new model URI.

mlflow deployments update -t ray-serve --name <deployment name> -C num_replicas=<new number of replicas>
Python API
plugin.update_deployment(name=<deployment name>, config={"num_replicas": <new number of replicas>})

Delete deployment

Delete an existing deployment.

mlflow deployments delete -t ray-serve --name <deployment name>
Python API
plugin.delete_deployment(name=<deployment name>)

List deployments

List the names of all the models deployed on Ray Serve. Includes models not deployed via this plugin.

mlflow deployments list -t ray-serve
Python API

Get deployment details

mlflow deployments get -t ray-serve --name <deployment name>
Python API
plugin.get_deployment(name=<deployment name>)

Run prediction on deployed model

For the prediction inputs, DataFrame, Tensor and JSON formats are supported by the Python API. To invoke via the command line, pass in the path to a JSON file containing the input.

mlflow deployments predict -t ray-serve --name <deployment name> --input-path <input file path> --output-path <output file path>

output-path is an optional parameter. Without it, the result will be printed in the terminal.

Python API
plugin.predict(name=<deployment name>, df=<prediction input>)

Plugin help

Prints the plugin help string.

mlflow deployments help -t ray-serve

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for mlflow-ray-serve, version 0.2.0
Filename, size File type Python version Upload date Hashes
Filename, size mlflow_ray_serve-0.2.0-py3-none-any.whl (10.0 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size mlflow-ray-serve-0.2.0.tar.gz (9.1 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page