sendnn-inference 2.0.0rc12

vLLM plugin for Spyre hardware support

SenDNN Inference

| Documentation | Users Forum | #sig-spyre |

---

IBM Spyre is the first production-grade Artificial Intelligence Unit (AIU) accelerator born out of the IBM Research AIU family, and is part of a long-term strategy of developing novel architectures and full-stack technology solutions for the emerging space of generative AI. Spyre builds on the foundation of IBM's internal AIU research and delivers a scalable, efficient architecture for accelerating AI in enterprise environments.

SenDNN Inference (sendnn-inference) is a vLLM plugin that enables seamless integration of IBM Spyre Accelerator with vLLM. It follows the architecture described in vLLM's Plugin System, making it easy to integrate IBM's advanced AI acceleration into existing vLLM workflows.

For more information, check out the following:

• 📚 Meet the IBM Artificial Intelligence Unit
• 📽️ AI Accelerators: Transforming Scalability & Model Efficiency
• 🚀 Spyre Accelerator for IBM Z
• 🚀 Spyre Accelerator for IBM POWER

Getting Started

Visit our documentation:

• Installation
• List of Supported Models
• List of Supported Features

Contributing

We welcome and value any contributions and collaborations. Please check out Contributing to SenDNN Inference for how to get involved.

Contact

You can reach out for discussion or support in the #sig-spyre channel in the vLLM Slack workspace or by opening an issue.