torchlambda 1588916857

Minimalistic & easy deployment of PyTorch models on AWS Lambda with C++

torchlambda is a tool to deploy PyTorch models on Amazon's AWS Lambda using AWS SDK for C++ and custom C++ runtime.

Using statically compiled dependencies whole package is shrunk to only 30MB.

Due to small size of compiled source code users can pass their models as AWS Lambda layers. Services like Amazon S3 are no longer necessary to load your model.

torchlambda has it's PyTorch & AWS dependencies always up to date because of continuous deployment run at 03:00 a.m. every day.

Docs	Deployment	Package	Python	PyTorch	Docker	CodeBeat	Images

Comparison with other deployment tools

• Do one thing and do it well - most deployment tools are complex solutions including multiple frameworks and multiple services. torchlambda focuses solely on PyTorch and AWS Lambda integration.
• Write programs to work together - Amazon AWS & PyTorch exist for a reason, no need to repeat their functionalities (like aws-cli). No source code modifications of your neural networks required either.
• Small is beautiful - 3000 LOC (most being convenience wrapper creating this tool) make it easy to delve into source code and modify what you want on your own.
• Integration with other tools - as torchlambda focuses on narrow space you can use any tools you like with PyTorch (e.g. for training you can use KubeFlow or BentoML) and AWS (for example Terraform).
• Easy to jump in - no need to learn new tool. torchlambda has at most 4 commands simplifying steps to take PyTorch model into the cloud which are mostly repetitive and possible to automate further.
• Extensible when you need it - All you usually need are a few lines of YAML settings, but if you wish to fine-tune your deployment you can use torchlambda build --flags (changing various properties of PyTorch and AWS dependencies).

Table Of Contents

• Installation
• Tutorials
  • ResNet18 deployment on AWS Lambda
  • Test Lambda deployment locally
  • base64 image encoding
• Commands
  • settings
  • template
  • build
  • layer
• YAML settings file reference
• C++ code

Footnotes

1. Support for latest version of it's main DL framework or main frameworks if multiple supported

2. Project dependencies are easily customizable. In torchlambda it would be user specified build procedures for libtorch and AWS C++ SDK

3. Necessary size of code and dependencies to deploy model

4. Based on Dockerfile size