bentoml

From a model in jupyter notebook to production API service in 5 minutes

BentoML is a python framework for serving and operating machine learning models, making it easy to promote trained models into high performance prediction services.

The framework provides high-level APIs for defining an ML service and packaging its trained model artifacts, preprocessing source code, dependencies, and configurations into a standard file format called BentoArchive - which can be deployed as REST API model server, PyPI package, CLI tool, or batch scoring job.

Check out our 5-mins quick start notebook using BentoML to productionize a scikit-learn model and deploy it to AWS Lambda.

Installation

pip install bentoml

Getting Started

Defining a machine learning service with BentoML is as simple as a few lines of code:

%%writefile keras_fashion_mnist.py

import numpy as np
from PIL import Image
from bentoml import api, artifacts, env, BentoService
from bentoml.artifact import KerasModelArtifact
from bentoml.handlers import ImageHandler

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

@env(conda_dependencies=['tensorflow', 'Pillow'])
@artifacts([KerasModelArtifact('classifier')])
class KerasFashionMnistService(BentoService):

    @api(ImageHandler, pilmode='L')
    def predict(self, img):
        img = Image.fromarray(img).resize((28, 28))
        img = np.array(img.getdata()).reshape((1,28,28,1))
        class_idx = self.artifacts.classifier.predict_classes(img)[0]
        return class_names[class_idx]

Import the defined BentoService, pack with trained model, and save a versioned archive to file system:

from keras_fashion_mnist import KerasFashionMnistService

model = keras.Sequential()
model.add(...)
...
model.compile(...)
model.fit(...)
model.evaluate(...)

# Packaging trained model for serving in production:
saved_path = KerasFashionMnistService.pack(classifier=model).save('/my_bento_archives')

Now you can start a REST API server based off the saved BentoArchive:

bentoml serve {saved_path}

Visit http://127.0.0.1:5000 in your browser to play around with the Web UI of the REST API model server, or try sending prediction request with curl:

curl -X POST "http://127.0.0.1:5000/predict" -H "Content-Type: image/png"
--data-binary @sample_image.png

The saved archive can also be used directly from CLI:

bentoml predict {saved_path} --input=sample_image.png

Or installed as a python package:

pip install {saved_path}

Try out the full example notebook here.

Feature Highlights

Multiple Distribution Format - Easily package your Machine Learning models and preprocessing code into a format that works best with your inference scenario:
- Docker Image - deploy as containers running REST API Server
- PyPI Package - integrate into your python applications seamlessly
- CLI tool - put your model into Airflow DAG or CI/CD pipeline
- Spark UDF - run batch serving on a large dataset with Spark
- Serverless Function - host your model on serverless platforms such as AWS Lambda
Multiple Framework Support - BentoML supports a wide range of ML frameworks out-of-the-box including Tensorflow, PyTorch, Keras, Scikit-Learn, xgboost, H2O, FastAI and can be easily extended to work with new or custom frameworks.
Deploy Anywhere - BentoML bundled ML service can be easily deployed with platforms such as Docker, Kubernetes, Serverless, Airflow and Clipper, on cloud platforms including AWS, Google Cloud, and Azure.
Custom Runtime Backend - Easily integrate your python pre-processing code with high-performance deep learning runtime backend, such as tensorflow-serving.

Documentation

Full documentation and API references can be found at bentoml.readthedocs.io

Examples

Quick Start Guide - Google Colab | nbviewer | source
Scikit-learn Sentiment Analysis - Google Colab | nbviewer | source
Keras Text Classification - Google Colab | nbviewer | source
Keras Fashion MNIST classification - Google Colab | nbviewer | source
FastAI Pet Classification - Google Colab | nbviewer | source
FastAI Tabular CSV - Google Colab | nbviewer | source
PyTorch Fashion MNIST classification - Google Colab | nbviewer | source
PyTorch CIFAR-10 Image classification - Google Colab | nbviewer | source
XGBoost Titanic Survival Prediction - Google Colab | nbviewer | source
H2O Classification- Google Colab | nbviewer | source

More examples can be found under the BentoML/examples directory or the bentoml/gallery repo.

Deployment guides:

Usage Tracking

BentoML library by default reports basic usages using Amplitude. It helps BentoML authors to understand how people are using this tool and improve it over time. You can easily opt-out by running the following command from terminal:

bentoml config set usage_tracking=false

Or from your python code:

import bentoml
bentoml.config.set('core', 'usage_tracking', 'false')

We also collect example notebook page views to help us understand the community interests. To opt-out of tracking, delete the ~~![Impression](http...~~ line in the first markdown cell of our example notebooks.

Contributing

Have questions or feedback? Post a new github issue or join our Slack chat room:

Want to help build BentoML? Check out our contributing guide and the development guide.

To make sure you have a pleasant experience, please read the code of conduct. It outlines core values and beliefs and will make working together a happier experience.

Happy hacking!

Releases

BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page. and follow the BentoML Community Calendar.

Watch BentoML Github repo for future releases:

gh-watch

License

Apache License 2.0

Release files for bentoml 0.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for bentoml 0.3.1
File	Size	Uploaded
BentoML-0.3.1.tar.gz	72.9 kB	Jul 25, 2019	Details

Built distribution (wheel)

Table of built distributions (wheels) for bentoml 0.3.1
File	Interpreter	ABI	Platform	Reset
BentoML-0.3.1-py3-none-any.whl 110.6 kB Jul 25, 2019	Python 3	none	any	Details

Total release size: 183.5 kB

Release files / BentoML-0.3.1.tar.gz

Download URL	BentoML-0.3.1.tar.gz
Size	72.9 kB
Tags	Source
SHA-256 checksum How to use checksums	`2b6da395c8cae987b3e11ae911fa03418c0053214440bc9dfa60939fe45dca3a`
BLAKE2b-256 checksum How to use checksums	`8592bc60aa9d28894e887c5e776d2d04d7a6829f03a7814321389e741afc228e`
Upload date	Jul 25, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.7.3`

Release files / BentoML-0.3.1-py3-none-any.whl

Download URL	BentoML-0.3.1-py3-none-any.whl
Size	110.6 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`b720b5fe2b8ba20155258ab29b09791ad91ccee1dd06862da6623d3915a9dea4`
BLAKE2b-256 checksum How to use checksums	`f2889c0c8cffba5e044ad83b25593a7fa92b08f8678ce965167d84edf561f115`
Upload date	Jul 25, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/40.6.3 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.7.3`