bentoml 0.3.1

A python framework for serving and operating machine learning models

From a model in jupyter notebook to production API service in 5 minutes

Installation | Getting Started | Documentation | Examples | Contributing | License

BentoML is a python framework for serving and operating machine learning models, making it easy to promote trained models into high performance prediction services.

The framework provides high-level APIs for defining an ML service and packaging its trained model artifacts, preprocessing source code, dependencies, and configurations into a standard file format called BentoArchive - which can be deployed as REST API model server, PyPI package, CLI tool, or batch scoring job.

Check out our 5-mins quick start notebook using BentoML to productionize a scikit-learn model and deploy it to AWS Lambda.

---

Installation

pip install bentoml

Getting Started

Defining a machine learning service with BentoML is as simple as a few lines of code:

%%writefile keras_fashion_mnist.py

import numpy as np
from PIL import Image
from bentoml import api, artifacts, env, BentoService
from bentoml.artifact import KerasModelArtifact
from bentoml.handlers import ImageHandler

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

@env(conda_dependencies=['tensorflow', 'Pillow'])
@artifacts([KerasModelArtifact('classifier')])
class KerasFashionMnistService(BentoService):

    @api(ImageHandler, pilmode='L')
    def predict(self, img):
        img = Image.fromarray(img).resize((28, 28))
        img = np.array(img.getdata()).reshape((1,28,28,1))
        class_idx = self.artifacts.classifier.predict_classes(img)[0]
        return class_names[class_idx]

Import the defined BentoService, pack with trained model, and save a versioned archive to file system:

from keras_fashion_mnist import KerasFashionMnistService

model = keras.Sequential()
model.add(...)
...
model.compile(...)
model.fit(...)
model.evaluate(...)

# Packaging trained model for serving in production:
saved_path = KerasFashionMnistService.pack(classifier=model).save('/my_bento_archives')

Now you can start a REST API server based off the saved BentoArchive:

bentoml serve {saved_path}

Visit http://127.0.0.1:5000 in your browser to play around with the Web UI of the REST API model server, or try sending prediction request with curl:

curl -X POST "http://127.0.0.1:5000/predict" -H "Content-Type: image/png"
--data-binary @sample_image.png

The saved archive can also be used directly from CLI:

bentoml predict {saved_path} --input=sample_image.png

Or installed as a python package:

pip install {saved_path}

Try out the full example notebook here.

Feature Highlights

• Multiple Distribution Format - Easily package your Machine Learning models and preprocessing code into a format that works best with your inference scenario:

  • Docker Image - deploy as containers running REST API Server
  • PyPI Package - integrate into your python applications seamlessly
  • CLI tool - put your model into Airflow DAG or CI/CD pipeline
  • Spark UDF - run batch serving on a large dataset with Spark
  • Serverless Function - host your model on serverless platforms such as AWS Lambda

• Multiple Framework Support - BentoML supports a wide range of ML frameworks out-of-the-box including Tensorflow, PyTorch, Keras, Scikit-Learn, xgboost, H2O, FastAI and can be easily extended to work with new or custom frameworks.

• Deploy Anywhere - BentoML bundled ML service can be easily deployed with platforms such as Docker, Kubernetes, Serverless, Airflow and Clipper, on cloud platforms including AWS, Google Cloud, and Azure.

• Custom Runtime Backend - Easily integrate your python pre-processing code with high-performance deep learning runtime backend, such as tensorflow-serving.

Documentation

Full documentation and API references can be found at bentoml.readthedocs.io

Examples

• Quick Start Guide - Google Colab | nbviewer | source
• Scikit-learn Sentiment Analysis - Google Colab | nbviewer | source
• Keras Text Classification - Google Colab | nbviewer | source
• Keras Fashion MNIST classification - Google Colab | nbviewer | source
• FastAI Pet Classification - Google Colab | nbviewer | source
• FastAI Tabular CSV - Google Colab | nbviewer | source
• PyTorch Fashion MNIST classification - Google Colab | nbviewer | source
• PyTorch CIFAR-10 Image classification - Google Colab | nbviewer | source
• XGBoost Titanic Survival Prediction - Google Colab | nbviewer | source
• H2O Classification- Google Colab | nbviewer | source

More examples can be found under the BentoML/examples directory or the bentoml/gallery repo.

Deployment guides:

• Serverless deployment with AWS Lambda
• API server deployment with AWS SageMaker
• API server deployment with Clipper
• API server deployment on Kubernetes

Usage Tracking

BentoML library by default reports basic usages using Amplitude. It helps BentoML authors to understand how people are using this tool and improve it over time. You can easily opt-out by running the following command from terminal:

bentoml config set usage_tracking=false

Or from your python code:

import bentoml
bentoml.config.set('core', 'usage_tracking', 'false')

We also collect example notebook page views to help us understand the community interests. To opt-out of tracking, delete the ![Impression](http... line in the first markdown cell of our example notebooks.

Contributing

Have questions or feedback? Post a new github issue or join our Slack chat room:

Want to help build BentoML? Check out our contributing guide and the development guide.

To make sure you have a pleasant experience, please read the code of conduct. It outlines core values and beliefs and will make working together a happier experience.

Happy hacking!

Releases

BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page. and follow the BentoML Community Calendar.

Watch BentoML Github repo for future releases:

License

Apache License 2.0