Skip to main content
Join the official 2020 Python Developers SurveyStart the survey!

A platform for serving and deploying machine learning models in the cloud

Project description

pypi status python versions Downloads build status Documentation Status join BentoML Slack

From ML model to production API endpoint with a few lines of code


BentoML makes it easy to serve and deploy machine learning models in the cloud.

It is an open source framework for building cloud-native model serving services. BentoML supports most popular ML training frameworks and deployment platforms, including major cloud providers and docker/kubernetes.

👉 Join BentoML Slack community to hear about the latest development updates.

Getting Started

Installing BentoML with pip:

pip install bentoml

Creating a prediction service with BentoML:

import bentoml
from bentoml.handlers import DataframeHandler
from bentoml.artifact import SklearnModelArtifact

@bentoml.env(pip_dependencies=["scikit-learn"]) # defining pip/conda dependencies to be packed
@bentoml.artifacts([SklearnModelArtifact('model')]) # defining required artifacts, typically trained models
class IrisClassifier(bentoml.BentoService):

    @bentoml.api(DataframeHandler) # defining prediction service endpoint and expected input format
    def predict(self, df):
        # Pre-processing logic and access to trained model artifacts in API function
        return self.artifacts.model.predict(df)

Train a classifier model with default Iris dataset and pack the trained model with the BentoService IrisClassifier defined above:

from sklearn import svm
from sklearn import datasets

if __name__ == "__main__":
    clf = svm.SVC(gamma='scale')
    iris = datasets.load_iris()
    X, y =,, y)

    # Create a iris classifier service
    iris_classifier_service = IrisClassifier()

    # Pack it with the newly trained model artifact
    iris_classifier_service.pack('model', clf)

    # Save the prediction service to a BentoService bundle
    saved_path =

You've just created a BentoService SavedBundle, it's a versioned file archive that is ready for production deployment. It contains the BentoService you defined, as well as the packed trained model artifacts, pre-processing code, dependencies and other configurations in a single file directory.

You can start a REST API server by specifying the BentoService's name and version, or provide the file path to the saved bundle:

bentoml serve IrisClassifier:latest
# or
bentoml serve {saved_path}

The REST API server provides web UI for testing and debugging the server. If you are running this command on your local machine, visit in your browser and try out sending API request to the server. You can also send prediction request with curl from command line:

curl -i \
  --header "Content-Type: application/json" \
  --request POST \
  --data '[[5.1, 3.5, 1.4, 0.2]]' \

The BentoService SavedBundle directory is structured to work as a docker build context, that can be used to build a API server docker container image: build context directory:

docker build -t my_api_server {saved_path}

You can also deploy your BentoService to cloud services such as AWS Lambda with bentoml lambda command. The deployment gives you a production-ready API endpoint hosting the BentoService you specified:

> bentoml get IrisClassifier
BENTO_SERVICE                         CREATED_AT        APIS                       ARTIFACTS
IrisClassifier:20200121114004_360ECB  2020-01-21 19:40  predict::DataframeHandler  model::SklearnModelArtifact
IrisClassifier:20200120082658_4169CF  2020-01-20 16:27  predict::DataframeHandler  clf::PickleArtifact

> bentoml lambda deploy test-deploy -b IrisClassifier:20200121114004_360ECB

More detailed code and walkthrough of this example can be found in the BentoML Quickstart Guide.


Full documentation and API references:


Visit bentoml/gallery repository for more examples and tutorials.




Tensorflow Keras

Tensorflow 2.0




Deployment guides:


Have questions or feedback? Post a new github issue or discuss in our Slack channel: join BentoML Slack

Want to help build BentoML? Check out our contributing guide and the development guide.


BentoML is under active development and is evolving rapidly. Currently it is a Beta release, we may change APIs in future releases.

Read more about the latest features and changes in BentoML from the releases page.

Usage Tracking

BentoML by default collects anonymous usage data using Amplitude. It only collects BentoML library's own actions and parameters, no user or model data will be collected. Here is the code that does it.

This helps BentoML team to understand how the community is using this tool and what to build next. You can easily opt-out of usage tracking by running the following command:

# From terminal:
bentoml config set usage_tracking=false
# From python:
import bentoml
bentoml.config().set('core', 'usage_tracking', 'False')


Apache License 2.0

FOSSA Status

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for BentoML, version 0.6.2
Filename, size File type Python version Upload date Hashes
Filename, size BentoML-0.6.2-py3-none-any.whl (554.4 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size BentoML-0.6.2.tar.gz (460.0 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page