torchlambda is a tool to deploy PyTorch models on Amazon's AWS Lambda using AWS SDK for C++ and custom C++ runtime.

Using statically compiled dependencies whole package is shrunk to only 30MB.

Due to small size of compiled source code users can pass their models as AWS Lambda layers. Services like Amazon S3 are no longer necessary to load your model.

torchlambda has it's PyTorch & AWS dependencies always tested & up to date because of daily continuous deployment runs.

Docs Deployment Package Python PyTorch Docker CodeBeat Images
Documentation CD PyPI Python PyTorch Docker codebeat badge Images

:heavy_check_mark: Why should I use torchlambda?

  • Lightweight & latest dependencies - compiled source code weights only 30MB. Previous approach to PyTorch network deployment on AWS Lambda (fastai) uses outdated PyTorch (1.1.0) as dependency layer and requires AWS S3 to host your model. Now you can only use AWS Lambda and host your model as layer and PyTorch master and latest stable release are supported on a daily basis.
  • Cheaper and less resource hungry - available solutions run server hosting incoming requests all the time. AWS Lambda (and torchlambda) runs only when the request comes.
  • Easy automated scaling usually autoscaling is done with Kubernetes or similar tools (see KubeFlow). This approach requires knowledge of another tool, setting up appropriate services (e.g. Amazon EKS). In AWS Lambda case you just push your neural network inference code and you are done.
  • Easy to use - no need to learn new tool. torchlambda has at most 4 commands and deployment is done via YAML settings. No need to modify your PyTorch code.
  • Do one thing and do it well - most deployment tools are complex solutions including multiple frameworks and multiple services. torchlambda focuses solely on inference of PyTorch models on AWS Lambda.
  • Write programs to work together - This tool does not repeat PyTorch & AWS's functionalities (like aws-cli). You can also use your favorite third party tools (say saws, Terraform with AWS and MLFlow, PyTorch-Lightning to train your model).
  • Test locally, run in the cloud - torchlambda uses Amazon Linux 2 Docker images under the hood & allows you to use lambci/docker-lambda to test your deployment on localhost before pushing deployment to the cloud (see Test Lambda deployment locally tutorial).
  • Extensible when you need it - All you usually need are a few lines of YAML settings, but if you wish to fine-tune your deployment you can use torchlambda build --flags (changing various properties of PyTorch and AWS dependencies themselves). You can also write your own C++ deployment code (generate template via torchlambda template command).
  • Small is beautiful - 3000 LOC (most being convenience wrapper creating this tool) make it easy to jump into source code and check what's going on under the hood.

:house: Table Of Contents

:page_with_curl: Benchmarks

Benchmarks can be seen in BENCHMARKS.md file and are comprised of around ~30000 test cases.

Results are divided based on settings used, model type, payload, AWS Lambda timing etc. Below is an example of how inference performance changes due to higher resolution images and type of encoding:

128x128 256x256 512x512 1024x1024
base64 120.622 165.184 311.129 995.249
byte 133.315 203.628 498.391 1738.97
char 128.331 209.306 517.482 1822.56
short 135.859 207.389 497.818 1740.91
int 133.42 216.163 519.502 1783.02
long 126.979 228.497 516.98 1760.93
float 135.825 223.045 515.245 1802.25
double 137.281 209.267 536.959 1811.83

Clearly the bigger image, the more important it is to use base64 encoding. For all results and description click here.

:question: Contributing

If you find an issue or you think some functionality may be useful to you, please open new Issue or create Pull Request with your changes, thanks!

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1602697918)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for szymonmaszke from gravatar.com szymonmaszke

Credits

Author: Szymon Maszke

License

MIT License (MIT)

Requires

Python >=3.6

Tags

aws lambda pytorch deployment minimal c++ neural network model

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

torchlambda-1595387628.tar.gz (26.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

torchlambda-1595387628-py3-none-any.whl (34.8 kB view details)

Uploaded Python 3

File details

Details for the file torchlambda-1595387628.tar.gz.

File metadata

  • Download URL: torchlambda-1595387628.tar.gz
  • Upload date:
  • Size: 26.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.2.0 requests-toolbelt/0.9.1 tqdm/4.48.0 CPython/3.8.5

File hashes

Hashes for torchlambda-1595387628.tar.gz
Algorithm Hash digest
SHA256 a10fc7d416a42976dcbc14baf9781cbe393ebadfc2a75de7c6c0da2a95ac0fd9
MD5 51d30f1c78ee45bf84b050fbe977cfcd
BLAKE2b-256 df5b8bf9c22a9993ec80bd54e08c701e243a7f2dde6f7b28c2aec39dd3b981a1

See more details on using hashes here.

File details

Details for the file torchlambda-1595387628-py3-none-any.whl.

File metadata

  • Download URL: torchlambda-1595387628-py3-none-any.whl
  • Upload date:
  • Size: 34.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/49.2.0 requests-toolbelt/0.9.1 tqdm/4.48.0 CPython/3.8.5

File hashes

Hashes for torchlambda-1595387628-py3-none-any.whl
Algorithm Hash digest
SHA256 1c1af983bafe53b52363dcf7d41c22b93dbde31530d12de0aacdaa456ce5596a
MD5 c80c73d803af1c73398243417d219b8e
BLAKE2b-256 58b489304264ba938046358243780cf996c9e403692e0cffec15e8c44c359692

See more details on using hashes here.

Release history Release notifications | RSS feed

1602697918

2 files

1601291906

2 files

1595906030

2 files

1595819652

2 files

1595733302

2 files

1595474049

2 files

This release

1595387628 This release

2 files

1595301221

2 files

1595214841

2 files

1595128522

2 files

1595041925

2 files

1594955646

2 files

1594869114

2 files

1594782747

2 files

1594696299

2 files

1594609980

2 files

1594523557

2 files

1594437091

2 files

1594350707

2 files

1594264306

2 files

1594177870

2 files

1594094149

2 files

1594005065

2 files

1593918755

2 files

1593832226

2 files

1593745839

2 files

1593659437

2 files

1593573080

2 files

1593486308

2 files

1593399935

2 files

1593313558

2 files

1593227076

2 files

1593140678

2 files

1593054270

2 files

1592967843

2 files

1592881427

2 files

1592795072

2 files

1592708716

2 files

1592622225

2 files

1592535849

2 files

1592449451

2 files

1592363000

2 files

1592276628

2 files

1592190229

2 files

1592103896

2 files

1592017417

2 files

1591931008

2 files

1591844533

2 files

1591758192

2 files

1591671785

2 files

1591585390

2 files

1591499021

2 files

1591412566

2 files

1591326285

2 files

1591239753

2 files

1591153498

2 files

1591066990

2 files

1591017093

2 files

1590990451

2 files

1590904228

2 files

1590817644

2 files

1590731191

2 files

1590644951

2 files

1590472189

2 files

1590385718

2 files

1590299449

2 files

1590212916

2 files

1590040004

2 files

1589867204

2 files

1589780740

2 files

1589694355

2 files

1589607984

2 files

1589521526

2 files

1589435214

2 files

1589348729

2 files

1589262226

2 files

1589175683

2 files

1589089467

2 files

1589003279

2 files

1588916857

2 files

1588830431

2 files

1588743886

2 files

1588657448

2 files

1588570997

2 files

1588484620

2 files

1588398086

2 files

1588311825

2 files

1588225513

2 files

1588052537

2 files

1587966020

2 files

1587793412

2 files

1587706841

2 files

1587447737

2 files

1587361328

2 files

1587188625

2 files

1587015624

2 files

1586929498

2 files

1586842928

2 files

1586317413

2 files

1586230943

2 files

1586030961

2 files

1585885600

2 files

1585813450

2 files

0.1.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page