bentoml 0.4.0

A python framework for serving and operating machine learning models

From a model in jupyter notebook to production API service in 5 minutes

Getting Started | Documentation | Examples | Contributing | Releases | License | Blog

BentoML is a platform for serving and deploying machine learning models, making it easy to productionize trained models.

BentoML framework provides:

• BentoService: High-level APIs for defining an ML service and packaging its trained model artifacts, preprocessing source code, dependencies, and configurations into a standard file format "Bento" that can be deployed as containerize REST API server, PyPI package, CLI tool, or batch/streaming inference job.

• Yatai: A stateful server that provides Web UI and APIs for accesing model registry on top of cloud storage and manages model serving deployments on cloud platforms such as AWS, Azure and GCP.

Check out the 5-mins quick start notebook using BentoML to productionize a scikit-learn model and deploy it to AWS Lambda:

---

Getting Started

Installation with pip:

pip install bentoml

Defining a machine learning service with BentoML:

import bentoml
from bentoml.artifact import PickleArtifact
from bentoml.handlers import DataframeHandler

# You can also import your own Python module here and BentoML will automatically
# figure out the dependency chain and package all those Python modules
import my_preproceesing_lib

@bentoml.artifacts([PickleArtifact('model')])
@bentoml.env(pip_dependencies=["scikit-learn"])
class IrisClassifier(bentoml.BentoService):

    @bentoml.api(DataframeHandler)
    def predict(self, df):
        # Preprocessing prediction request - DataframeHandler parses REST API
        # request or CLI args into pandas Dataframe that can be easily processed
        # into feature vectors that are ready for the trained model
        df = my_preproceesing_lib.process(df)

        # Assess to serialized trained model artifact via self.artifacts
        return self.artifacts.model.predict(df)

After training your ML model, you can pack it with the prediction service IrisClassifier defined above, and save them as a Bento to file system:

from sklearn import svm
from sklearn import datasets

clf = svm.SVC(gamma='scale')
iris = datasets.load_iris()
X, y = iris.data, iris.target
clf.fit(X, y)

# Packaging trained model for serving in production:
saved_path = IrisClassifier.pack(model=clf).save('/tmp/bento')

A Bento is a versioned archive, containing the BentoService you defined, along with trained model artifacts, dependencies and configurations etc. BentoML library can then load in a Bento file and turn it into a high performance prediction service.

For example, you can now start a REST API server based off the saved Bento files:

bentoml serve {saved_path}

Visit http://127.0.0.1:5000 in your browser to play around with the Web UI of the REST API model server, sending testing requests from the UI, or try sending prediction request with curl from CLI:

curl -i \
  --header "Content-Type: application/json" \
  --request POST \
  --data '[[5.1, 3.5, 1.4, 0.2]]' \
  http://localhost:5000/predict

The saved archive can also be used directly from CLI:

bentoml predict {saved_path} --input='[[5.1, 3.5, 1.4, 0.2]]'

# alternatively:
bentoml predict {saved_path} --input='./iris_test_data.csv'

Saved Bento can also be installed and used as a Python PyPI package:

pip install {saved_path}

# Your bentoML model class name will become packaged name
import IrisClassifier

installed_svc = IrisClassifier.load()
installed_svc.predict([[5.1, 3.5, 1.4, 0.2]])

You can also build a docker image for this API server with all dependencies and environments configured automatically by BentoML, and share the docker image with your DevOps team for deployment in production:

docker build -t my_api_server {saved_path}

Try out the full getting started notebook here on Google Colab.

Examples

FastAI

• Pet Image Classification - Google Colab | nbviewer | source
• Salary Range Prediction - Google Colab | nbviewer | source

Scikit-Learn

• Sentiment Analysis - Google Colab | nbviewer | source
• Loan Prediction - Google Colab | nbviewer | source

PyTorch

• Fashion MNIST - Google Colab | nbviewer | source
• CIFAR-10 Image Classification - Google Colab | nbviewer | source

Tensorflow Keras

• Fashion MNIST - Google Colab | nbviewer | source
• Text Classification - Google Colab | nbviewer | source
• Toxic Comment Classifier - Google Colab | nbviewer | source

XGBoost

• Titanic Survival Prediction - Google Colab | nbviewer | source
• League of Legend win Prediction - Google Colab | nbviewer | source

H2O

• Loan Default Prediction - Google Colab | nbviewer | source
• Prostate Cancer Prediction - Google Colab | nbviewer | source

Visit bentoml/gallery repository for more example projects demonstrating how to use BentoML:

Deployment guides:

• Serverless deployment with AWS Lambda
• API server deployment with AWS SageMaker
• API server deployment with Clipper
• API server deployment on Kubernetes

Feature Highlights

• Multiple Distribution Format - Easily package your Machine Learning models and preprocessing code into a format that works best with your inference scenario:

  • Docker Image - deploy as containers running REST API Server
  • PyPI Package - integrate into your python applications seamlessly
  • CLI tool - put your model into Airflow DAG or CI/CD pipeline
  • Spark UDF - run batch serving on a large dataset with Spark
  • Serverless Function - host your model on serverless platforms such as AWS Lambda

• Multiple Framework Support - BentoML supports a wide range of ML frameworks out-of-the-box including Tensorflow, PyTorch, Keras, Scikit-Learn, xgboost, H2O, FastAI and can be easily extended to work with new or custom frameworks.

• Deploy Anywhere - BentoML bundled ML service can be easily deployed with platforms such as Docker, Kubernetes, Serverless, Airflow and Clipper, on cloud platforms including AWS, Google Cloud, and Azure.

• Custom Runtime Backend - Easily integrate your python pre-processing code with high-performance deep learning runtime backend, such as tensorflow-serving.

Documentation

Full documentation and API references can be found at bentoml.readthedocs.io

Usage Tracking

BentoML library by default reports basic usages using Amplitude. It helps BentoML authors to understand how people are using this tool and improve it over time. You can easily opt-out by running the following command from terminal:

bentoml config set usage_tracking=false

Or from your python code:

import bentoml
bentoml.config.set('core', 'usage_tracking', 'false')

We also collect example notebook page views to help us understand the community interests. To opt-out of tracking, delete the ![Impression](http... line in the first markdown cell of our example notebooks.

Contributing

Have questions or feedback? Post a new github issue or join our Slack chat room:

Want to help build BentoML? Check out our contributing guide and the development guide.

To make sure you have a pleasant experience, please read the code of conduct. It outlines core values and beliefs and will make working together a happier experience.

Happy hacking!

Releases

BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page. and follow the BentoML Community Calendar.

Watch BentoML Github repo for future releases:

License

Apache License 2.0