torch-nos 0.0.10

Nitrous oxide system (NOS) for PyTorch.

🔥 NOS

Optimize, serve and auto-scale Pytorch models on any hardware.
Cut your inference costs by 10x.

Website | Docs | Discord

NOS is a PyTorch library for optimizing and running lightning-fast inference of popular computer vision models.

Optimizing and serving models for production AI inference is still difficult, often leading to notoriously expensive cloud bills and often underutilized GPUs. That’s why we’re building NOS - a fast inference server for modern AI workloads. With a few lines of code, developers can optimize, serve, and auto-scale Pytorch model inference without having to deal with the complexities of ML compilers, HW-accelerators, or distributed inference. Simply put, NOS allows AI teams to cut inference costs up to 10x, speeding up development time and time-to-market.

What is NOS?

• ⚡️ Fast: Built for PyTorch and designed to optimize/run models faster
• 🔥 Performant: Run models such as SDv2 or object detection 2-3x faster out-of-the-box
• 👩‍💻 No PhD required: Optimize models for maximum HW performance without a PhD in ML
• 📦 Extensible: Easily add optimization and HW-support for custom models
• ⚙️ HW-accelerated: Take full advantage of your HW (GPUs, ASICs) without compromise
• ☁️ Cloud-agnostic: Run on any cloud HW (AWS, GCP, Azure, Lambda Labs, On-Prem)

NOS inherits its name from Nitrous Oxide System, the performance-enhancing system typically used in racing cars. NOS is designed to be modular and easy to extend.

Batteries Included

• 💪 SOTA Model Support: NOS provides out-of-the-box support for popular CV models such as Stable Diffusion, OpenAI CLIP, YOLOX object detection, tracking and more
• 🔌 APIs: NOS provides out-of-the-box APIs and avoids all the ML model deployment hassles
• 🐳 Docker: NOS ships with docker images to run accelerated and scalable CV workloads
• 📈 Multi-Platform: NOS allows you to run models on different HW (NVIDIA, custom ASICs) without any model compilation or runtime management.

Getting Started

Get started with NOS in a few lines of code:

pip install torch-nos[torch]

• API Documentation
• Quickstart
• Supported Models
• Roadmap

Contribute

We welcome contributions! Please see our contributing guide for more information.