torchlambda 1587015624

Minimalistic & easy deployment of PyTorch models on AWS Lambda with C++

torchlambda is a tool to deploy PyTorch models on Amazon's AWS Lambda using AWS SDK for C++ and custom C++ runtime.

Using statically compiled dependencies whole package is shrunk to only 30MB.

Due to small size of compiled source code users can pass their models as AWS Lambda layers. Services like Amazon S3 are no longer necessary to load your model.

torchlambda has it's PyTorch & AWS dependencies always up to date because of continuous deployment run at 03:00 a.m. every day.

Docs	Deployment	Package	Python	PyTorch	Docker	CodeBeat	Images

Comparison with other deployment tools

Improve this comparison's reliability via Pull Request, thanks. Also show guys below some love by visiting their projects (just click on the name).

Trait / Tool	torchlambda	fastai Lambda	KubeFlow	Tensorflow Serving
Autoscaling	:heavy_check_mark:	:heavy_check_mark:	with Kubernetes	with Kubernetes
Light/Heavy load	Light	Light	Heavy/Both	Both
GPU Support	:x:	:x:	:heavy_check_mark:	:heavy_check_mark:
Serverless	:heavy_check_mark:	:heavy_check_mark:	:x:	:x:
Required services	AWS Lambda	AWS Lambda, AWS S3	Kubernetes Cluster & cloud provider	Deployable in various settings
Multiple frameworks	:x:	:x:	:heavy_check_mark:	:x:
Latest framework 1	:heavy_check_mark:	:x:	:x:	:heavy_check_mark:
Version (higher more mature)	CD	N/A	1.0	2.1.0
Customizable dependencies 2	:heavy_check_mark:	:x:	:x:	:x:
Deployment size 3	~30Mb	+1Gb	N/A	~67Mb4

Table Of Contents

• Installation

• Tutorials

  • Basic deployment on AWS Lambda
  • base64 image encoding
  • Testing your function locally

• Commands

  • settings
  • template
  • build
  • layer

• YAML settings file reference

• C++ code

Footnotes

1. Support for latest version of it's main DL framework or main frameworks if multiple supported

2. Project dependencies are easily customizable. In torchlambda it would be user specified build procedures for libtorch and AWS C++ SDK

3. Necessary size of code and dependencies to deploy model

4. Based on Dockerfile size