Nitrous oxide system (NOS) for computer-vision.
Project description
🔥 NOS
Optimize, serve and auto-scale Pytorch models on any hardware.
Cut your inference costs by 10x.
NOS is a PyTorch library for optimizing and running lightning-fast inference of popular computer vision models.
Optimizing and serving models for production AI inference is still difficult, often leading to notoriously expensive cloud bills and often underutilized GPUs. That’s why we’re building NOS - a fast inference server for modern AI workloads. With a few lines of code, developers can optimize, serve, and auto-scale Pytorch model inference without having to deal with the complexities of ML compilers, HW-accelerators, or distributed inference. Simply put, NOS allows AI teams to cut inference costs up to 10x, speeding up development time and time-to-market.
What is NOS?
- ⚡️ Fast: Built for PyTorch and designed to optimize/run models faster
- 🔥 Performant: Run models such as SDv2 or object detection 2-3x faster out-of-the-box
- 👩💻 No PhD required: Optimize models for maximum HW performance without a PhD in ML
- 📦 Extensible: Easily add optimization and HW-support for custom models
- ⚙️ HW-accelerated: Take full advantage of your HW (GPUs, ASICs) without compromise
- ☁️ Cloud-agnostic: Run on any cloud HW (AWS, GCP, Azure, Lambda Labs, On-Prem)
NOS inherits its name from Nitrous Oxide System, the performance-enhancing system typically used in racing cars. NOS is designed to be modular and easy to extend.
Batteries Included
- 💪 SOTA Model Support: NOS provides out-of-the-box support for popular CV models such as Stable Diffusion, OpenAI CLIP, YOLOX object detection, tracking and more
- 🔌 APIs: NOS provides out-of-the-box APIs and avoids all the ML model deployment hassles
- 🐳 Docker: NOS ships with docker images to run accelerated and scalable CV workloads
- 📈 Multi-Platform: NOS allows you to run models on different HW (NVIDIA, custom ASICs) without any model compilation or runtime management.
Getting Started
Get started with NOS in a few lines of code:
pip install autonomi-nos[torch]
Contribute
We welcome contributions! Please see our contributing guide for more information.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
File details
Details for the file autonomi_nos-0.0.10-py3-none-any.whl
.
File metadata
- Download URL: autonomi_nos-0.0.10-py3-none-any.whl
- Upload date:
- Size: 103.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a20a9359939b4280ae06c58953361bbc65e90bc13f4817e49a6205ea8e8f5487 |
|
MD5 | 9fa63fdfd7bc232d665cfec72a9ffad7 |
|
BLAKE2b-256 | fdc9fc53d0696041ccd0be57ee7fa702219a67fd857289e8692d84cf22592743 |