bentoml

From a model in jupyter notebook to production API service in 5 minutes

BentoML accelerates the workflow of serving and deploying machine learning models in the cloud for data scientists.

It is a flexible framework supporting most popular machine learning frameworks and making it easy for machine learning teams to deliver cloud-native prediction services that DevOps and software engineers can easily work with. Check out our 5-mins Quickstart Notebook using BentoML to turn a trained sklearn model into a containerized REST API server, and then deploy it to AWS Lambda.

If you plan to use BentoML for production workloads or want to contribute, be sure to join our Slack channel and hear about our latest development updates:

Getting Started

Installation with pip:

pip install bentoml

Defining a prediction service with BentoML:

import bentoml
from bentoml.handlers import DataframeHandler
from bentoml.artifact import SklearnModelArtifact

@bentoml.env(pip_dependencies=["scikit-learn"]) # defining pip/conda dependencies to be packed
@bentoml.artifacts([SklearnModelArtifact('model')]) # defining required artifacts, typically trained models
class IrisClassifier(bentoml.BentoService):

    @bentoml.api(DataframeHandler) # defining prediction service endpoint and expected input format
    def predict(self, df):
        # Pre-processing logic and access to trained mdoel artifacts in API function
        return self.artifacts.model.predict(df)

Train a classifier model with default Iris dataset and pack the trained model with the BentoService IrisClassifier defined above:

from sklearn import svm
from sklearn import datasets

if __name__ == "__main__":
    clf = svm.SVC(gamma='scale')
    iris = datasets.load_iris()
    X, y = iris.data, iris.target
    clf.fit(X, y)

    # Create a iris classifier service
    iris_classifier_service = IrisClassifier()

    # Pack it with the newly trained model artifact
    iris_classifier_service.pack('model', clf)

    # Save the prediction service to a BentoService bundle
    saved_path = iris_classifier_service.save()

A BentoService bundle is a versioned file archive, containing the BentoService you defined, along with trained model artifacts, dependencies and configurations.

Now you can start a REST API server based off the saved BentoService bundle form command line:

bentoml serve {saved_path}

If you are doing this only local machine, visit http://127.0.0.1:5000 in your browser to play around with the API server's Web UI for debbugging and sending test request. You can also send prediction request with curl from command line:

curl -i \
  --header "Content-Type: application/json" \
  --request POST \
  --data '[[5.1, 3.5, 1.4, 0.2]]' \
  http://localhost:5000/predict

The saved BentoService bundle can also be loaded directly from command line for inferencing:

bentoml predict {saved_path} --input='[[5.1, 3.5, 1.4, 0.2]]'

# alternatively:
bentoml predict {saved_path} --input='./iris_test_data.csv'

BentoService bundle is pip-installable and can be directly distributed as a PyPI package:

pip install {saved_path}

# Your bentoML model class name will become packaged name
import IrisClassifier

installed_svc = IrisClassifier.load()
installed_svc.predict([[5.1, 3.5, 1.4, 0.2]])

BentoService bundle is also structured to work as a docker build context, which can be used to build a docker image containing the REST API model server:

docker build -t my_api_server {saved_path}

To learn more, try out the Getting Started with Bentoml notebook:

Examples

FastAI

Pet Image Classification - Google Colab | nbviewer | source
Salary Range Prediction - Google Colab | nbviewer | source

Scikit-Learn

Sentiment Analysis - Google Colab | nbviewer | source

PyTorch

Fashion MNIST - Google Colab | nbviewer | source
CIFAR-10 Image Classification - Google Colab | nbviewer | source

Keras

Fashion MNIST - Google Colab | nbviewer | source
Text Classification - Google Colab | nbviewer | source
Toxic Comment Classifier - Google Colab | nbviewer | source

XGBoost

Titanic Survival Prediction - Google Colab | nbviewer | source
League of Legend win Prediction - Google Colab | nbviewer | source

H2O

Loan Default Prediction - Google Colab | nbviewer | source
Prostate Cancer Prediction - Google Colab | nbviewer | source

Visit bentoml/gallery repository for more example projects demonstrating how to use BentoML.

Deployment guides:

Feature Highlights

Multiple Distribution Format - Easily package your Machine Learning models and preprocessing code into a format that works best with your inference scenario:
- Docker Image - deploy as containers running REST API Server
- PyPI Package - integrate into your python applications seamlessly
- CLI tool - put your model into Airflow DAG or CI/CD pipeline
- Spark UDF - run batch serving on a large dataset with Spark
- Serverless Function - host your model on serverless platforms such as AWS Lambda
Multiple Framework Support - BentoML supports a wide range of ML frameworks out-of-the-box including Tensorflow, PyTorch, Keras, Scikit-Learn, xgboost, H2O, FastAI and can be easily extended to work with new or custom frameworks
Deploy Anywhere - BentoService bundle can be easily deployed with platforms such as Docker, Kubernetes, Serverless, Airflow and Clipper, on cloud platforms including AWS, Google Cloud, and Azure
Custom Runtime Backend - Easily integrate your python pre-processing code with high-performance deep learning runtime backend, such as tensorflow-serving
Workflow Designed For Teams - The YataiService component in BentoML provides Web UI and APIs for managing and deploying all the models and preidciton services your team has created or deployed, in a centralized service.

Documentation

Full documentation and API references can be found at bentoml.readthedocs.io

Usage Tracking

BentoML library by default reports basic usages using Amplitude. It helps BentoML authors to understand how people are using this tool and improve it over time. You can easily opt-out by running the following command from terminal:

bentoml config set usage_tracking=false

Contributing

Have questions or feedback? Post a new github issue or join our Slack channel:

Want to help build BentoML? Check out our contributing guide and the development guide.

Releases

BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page.

License

Apache License 2.0

Release files for bentoml 0.4.9

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for bentoml 0.4.9
File	Size	Uploaded
BentoML-0.4.9.tar.gz	440.7 kB	Nov 11, 2019	Details

Built distribution (wheel)

Table of built distributions (wheels) for bentoml 0.4.9
File	Interpreter	ABI	Platform	Reset
BentoML-0.4.9-py3-none-any.whl 493.1 kB Nov 11, 2019	Python 3	none	any	Details

Total release size: 933.9 kB

Release files / BentoML-0.4.9.tar.gz

Download URL	BentoML-0.4.9.tar.gz
Size	440.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`701c4b3836af96f11fd5659c9ae160eacbaa011949b1e3b1273e589fc3cb41b8`
BLAKE2b-256 checksum How to use checksums	`ca0032657d4f6e75177bfbf0f01a59a70054122b8aae28cfc669f726cae7a425`
Upload date	Nov 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.7.3`

Release files / BentoML-0.4.9-py3-none-any.whl

Download URL	BentoML-0.4.9-py3-none-any.whl
Size	493.1 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`3adb2afef407c922bd8557eaadcf7f9438b54a77795d9414e692b96ef45b6175`
BLAKE2b-256 checksum How to use checksums	`f1ffa38b769091072e72d47c6316e482226bde305b7730b7a6665b3783f47b15`
Upload date	Nov 11, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.4.0 requests-toolbelt/0.8.0 tqdm/4.29.1 CPython/3.7.3`