Open source toolkit for helping create serving containers to run on Amazon SageMaker.
Project description
SageMaker Inference Toolkit is a library used for enabling serving within the SageMaker prebuilt deep learning framework containers.
This library is the serving subset of the SageMaker Containers library.
Currently, this library is used by the following containers:
Table of Contents
Getting Started
The main purpose of this library is to start up a model server within a container to enable serving on SageMaker.
This library assumes the following SageMaker inference requirements are met.
The following code block shows how to start the model server.
from sagemaker_inference import model_server
model_server.start_model_server(handler_service=HANDLER_SERVICE)
The HANDLER_SERVICE is a string literal that points to the Python path of a Python file that will be executed by the model server for incoming invocation requests. This Python script is responsible for handling incoming data and passing it on to the engine for inference. The Python file should define a handle method that acts as an entry point for execution, this function will be invoked by the model server on a inference request.
For more information on how to define your HANDLER_SERVICE file, see Custom Service.
Running tests
To run all tests:
tox
License
This library is licensed under the Apache 2.0 License. It is copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved. The license is available at: http://aws.amazon.com/apache2.0/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for sagemaker_inference-1.0.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1abcadf16c32f5c7322ee9764970acaf2552c2fce2c64ef74701721df801bdba |
|
MD5 | 73150540ba05e133659def2d5c4e4307 |
|
BLAKE2b-256 | b0e4ef45abbaa2c522b7779a7927e8e7a0a413b501a6733dbe65e8e0f5512bab |