vllm-rbln 0.10.4a1

vLLM plugin for RBLN NPU

vLLM RBLN Plugin

This repository provides the hardware plugin that enables vLLM on RBLN NPUs, including ATOM and REBEL.

Built on top of vLLM’s Plugin System, it allows seamless integration with the vLLM ecosystem and provides high-throughput, low-latency LLM serving on RBLN hardware. Our plugin supports a wide range of popular LLMs and continues to expand to support all features enabled in vLLM, including advanced attention mechanisms.

🚀 Getting Started

📋 Prerequisites

• rebel-compiler
• optimum-rbln

⚙️ Installation

You can install this project using pip or from source.

Install via PyPI

Using uv

uv pip install vllm-rbln --extra-index-url https://wheels.vllm.ai/0.18.0/cpu --torch-backend cpu

Using pip

pip install vllm-rbln --extra-index-url https://wheels.vllm.ai/0.18.0/cpu --extra-index-url https://download.pytorch.org/whl/cpu

Or from source

Using uv

git clone https://github.com/rbln-sw/vllm-rbln.git
cd vllm-rbln
uv pip install -e .

Using pip

git clone https://github.com/rbln-sw/vllm-rbln.git
cd vllm-rbln
pip install -e . --extra-index-url https://wheels.vllm.ai/0.18.0/cpu --extra-index-url https://download.pytorch.org/whl/cpu

📚 Documentation

• Overview & Supported Models
• API Tutorial

🤝 Contributing

We welcome all contributions! Whether it's reporting issues, proposing enhancements, or improving docs—your input helps make the project better.

See our CONTRIBUTING.md for more information.

📄 License

This project is licensed under the Apache License 2.0.

See the LICENSE file for more information.

📧 Contact

• Join discussions and get answers in our Developer Community
• Contact maintainers at [e-mail]