Skip to main content

Nitrous oxide system (NOS) for computer-vision.

Project description

🔥 NOS

Optimize, serve and auto-scale Pytorch models on any hardware.
Cut your inference costs by 10x.

Website | Docs | Discord

PyPi Version PyPi Version PyPi Downloads Discord PyPi Version

NOS is a PyTorch library for optimizing and running lightning-fast inference of popular computer vision models.

Optimizing and serving models for production AI inference is still difficult, often leading to notoriously expensive cloud bills and often underutilized GPUs. That’s why we’re building NOS - a fast inference server for modern AI workloads. With a few lines of code, developers can optimize, serve, and auto-scale Pytorch model inference without having to deal with the complexities of ML compilers, HW-accelerators, or distributed inference. Simply put, NOS allows AI teams to cut inference costs up to 10x, speeding up development time and time-to-market.

What is NOS?

  • ⚡️ Fast: Built for PyTorch and designed to optimize/run models faster
  • 🔥 Performant: Run models such as SDv2 or object detection 2-3x faster out-of-the-box
  • 👩‍💻 No PhD required: Optimize models for maximum HW performance without a PhD in ML
  • 📦 Extensible: Easily add optimization and HW-support for custom models
  • ⚙️ HW-accelerated: Take full advantage of your HW (GPUs, ASICs) without compromise
  • ☁️ Cloud-agnostic: Run on any cloud HW (AWS, GCP, Azure, Lambda Labs, On-Prem)

NOS inherits its name from Nitrous Oxide System, the performance-enhancing system typically used in racing cars. NOS is designed to be modular and easy to extend.

Batteries Included

  • 💪 SOTA Model Support: NOS provides out-of-the-box support for popular CV models such as Stable Diffusion, OpenAI CLIP, YOLOX object detection, tracking and more
  • 🔌 APIs: NOS provides out-of-the-box APIs and avoids all the ML model deployment hassles
  • 🐳 Docker: NOS ships with docker images to run accelerated and scalable CV workloads
  • 📈 Multi-Platform: NOS allows you to run models on different HW (NVIDIA, custom ASICs) without any model compilation or runtime management.

Getting Started

Get started with NOS in a few lines of code:

pip install autonomi-nos[torch]

Contribute

We welcome contributions! Please see our contributing guide for more information.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

autonomi_nos-0.0.10-py3-none-any.whl (103.1 kB view details)

Uploaded Python 3

File details

Details for the file autonomi_nos-0.0.10-py3-none-any.whl.

File metadata

File hashes

Hashes for autonomi_nos-0.0.10-py3-none-any.whl
Algorithm Hash digest
SHA256 a20a9359939b4280ae06c58953361bbc65e90bc13f4817e49a6205ea8e8f5487
MD5 9fa63fdfd7bc232d665cfec72a9ffad7
BLAKE2b-256 fdc9fc53d0696041ccd0be57ee7fa702219a67fd857289e8692d84cf22592743

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page