Skip to main content
Join the official 2020 Python Developers SurveyStart the survey!

A python framework for serving and operating machine learning models

Project description

pypi status python versions Downloads build status Documentation Status join BentoML Slack

From a model in jupyter notebook to production API service in 5 minutes


Installation | Getting Started | Documentation | Examples | Contributing | License

BentoML is a python framework for serving and operating machine learning models, making it easy to promote trained models into high performance prediction services.

The framework provides high-level APIs for defining an ML service and packaging its trained model artifacts, preprocessing source code, dependencies, and configurations into a standard file format called BentoArchive - which can be deployed as REST API model server, PyPI package, CLI tool, or batch scoring job.

Check out our 5-mins quick start notebook Google Colab Badge using BentoML to productionize a scikit-learn model and deploy it to AWS Lambda.


pypi status python versions

pip install bentoml

Getting Started

Defining a machine learning service with BentoML is as simple as a few lines of code:


import numpy as np
from PIL import Image
from bentoml import api, artifacts, env, BentoService
from bentoml.artifact import KerasModelArtifact
from bentoml.handlers import ImageHandler

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

@env(conda_dependencies=['tensorflow', 'Pillow'])
class KerasFashionMnistService(BentoService):

    @api(ImageHandler, pilmode='L')
    def predict(self, img):
        img = Image.fromarray(img).resize((28, 28))
        img = np.array(img.getdata()).reshape((1,28,28,1))
        class_idx = self.artifacts.classifier.predict_classes(img)[0]
        return class_names[class_idx]

Import the defined BentoService, pack with trained model, and save a versioned archive to file system:

from keras_fashion_mnist import KerasFashionMnistService

model = keras.Sequential()

# Packaging trained model for serving in production:
saved_path = KerasFashionMnistService.pack(classifier=model).save('/my_bento_archives')

Now you can start a REST API server based off the saved BentoArchive:

bentoml serve {saved_path}

Visit in your browser to play around with the Web UI of the REST API model server, or try sending prediction request with curl:

curl -X POST "" -H "Content-Type: image/png"
--data-binary @sample_image.png

The saved archive can also be used directly from CLI:

bentoml predict {saved_path} --input=sample_image.png

Or installed as a python package:

pip install {saved_path}

Try out the full example notebook here.

Feature Highlights

  • Multiple Distribution Format - Easily package your Machine Learning models and preprocessing code into a format that works best with your inference scenario:

    • Docker Image - deploy as containers running REST API Server
    • PyPI Package - integrate into your python applications seamlessly
    • CLI tool - put your model into Airflow DAG or CI/CD pipeline
    • Spark UDF - run batch serving on a large dataset with Spark
    • Serverless Function - host your model on serverless platforms such as AWS Lambda
  • Multiple Framework Support - BentoML supports a wide range of ML frameworks out-of-the-box including Tensorflow, PyTorch, Keras, Scikit-Learn, xgboost, H2O, FastAI and can be easily extended to work with new or custom frameworks.

  • Deploy Anywhere - BentoML bundled ML service can be easily deployed with platforms such as Docker, Kubernetes, Serverless, Airflow and Clipper, on cloud platforms including AWS, Google Cloud, and Azure.

  • Custom Runtime Backend - Easily integrate your python pre-processing code with high-performance deep learning runtime backend, such as tensorflow-serving.


Full documentation and API references can be found at


More examples can be found under the BentoML/examples directory or the bentoml/gallery repo.

Deployment guides:

Usage Tracking

BentoML library by default reports basic usages using Amplitude. It helps BentoML authors to understand how people are using this tool and improve it over time. You can easily opt-out by running the following command from terminal:

bentoml config set usage_tracking=false

Or from your python code:

import bentoml
bentoml.config.set('core', 'usage_tracking', 'false')

We also collect example notebook page views to help us understand the community interests. To opt-out of tracking, delete the ![Impression](http... line in the first markdown cell of our example notebooks.


Have questions or feedback? Post a new github issue or join our Slack chat room: join BentoML Slack

Want to help build BentoML? Check out our contributing guide and the development guide.

To make sure you have a pleasant experience, please read the code of conduct. It outlines core values and beliefs and will make working together a happier experience.

Happy hacking!


BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page. and follow the BentoML Community Calendar.

Watch BentoML Github repo for future releases:



Apache License 2.0

FOSSA Status

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for BentoML, version 0.3.1
Filename, size File type Python version Upload date Hashes
Filename, size BentoML-0.3.1-py3-none-any.whl (110.6 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size BentoML-0.3.1.tar.gz (72.9 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page