vllm-xpu-kernels

A vLLM component that provides optimized custom kernels for Intel GPUs (XPU) to accelerate LLM inference.

Table of Contents

• About
• Supported Kernels
• Requirements
• Getting Started
  • How It Works
  • Installation
  • Build from Source
  • Build Options
  • Using with vLLM
  • Kernel Configuration
• Testing
• Benchmarks
• License

About

vLLM defines and implements many custom Torch ops and kernels. This repository provides custom implementations for the Intel XPU (GPU) backend, enabling high-throughput LLM inference on Intel hardware.

Kernels are written in SYCL/DPC++ and leverage oneDNN for deep learning primitives. The library follows the PyTorch custom op registration and dispatch pattern — importing it at startup registers all ops for seamless use within vLLM.

Supported Kernels

Requirements

• Python: 3.12
• PyTorch: 2.14.0+xpu
• oneAPI: 2026.1 (Base Toolkit download)
• CMake: ≥ 3.26
• Ninja build system

Getting Started

How It Works

vLLM calls import vllm_xpu_kernels._C at startup, which registers all custom ops into the PyTorch dispatcher. From that point on, XPU ops are dispatched automatically whenever vLLM runs on Intel GPU hardware — no additional code changes are required in vLLM itself.

Installation

With a compatible PyTorch XPU installation already available, install the published package from PyPI into your active environment:

pip install --upgrade vllm-xpu-kernels

To pin a specific release, for example:

pip install "vllm-xpu-kernels==0.1.15.4"

Build from Source

1. Clone the repository

Run these commands on the host before building the Docker image or setting up a bare-metal build:

git clone https://github.com/vllm-project/vllm-xpu-kernels.git
cd vllm-xpu-kernels

2. Prepare oneAPI

• Option 1: Docker Container

  Build the development image using this repository's Dockerfile.xpu. Configure proxies as needed for your network.

  # Clone the repository and enter its directory on the host before building the Docker image.
  
  git clone https://github.com/vllm-project/vllm-xpu-kernels.git
  cd vllm-xpu-kernels
  docker build --no-cache \
               -f ./Dockerfile.xpu \
               -t vllm/vllm-xpu-kernels:latest .
  

  Launch the container with the checkout mounted as its working directory:

  docker run -it \
             -v /dev/dri/by-path:/dev/dri/by-path \
             -v "$(pwd):/workspace/vllm-xpu-kernels" \
             --device=/dev/dri \
             --ipc=host \
             --workdir /workspace/vllm-xpu-kernels \
             --name vllm-xpu-kernels \
             --entrypoint /bin/bash \
             vllm/vllm-xpu-kernels:latest
  

• Option 2: Bare-metal

  Install the Intel oneAPI Base Toolkit matching the source-build requirements above.

Initialize the oneAPI environment in the shell where you will build:

source /opt/intel/oneapi/setvars.sh

3. Install dependencies

In either environment, install the dependencies from the mounted or cloned repository:

pip install -r requirements.txt

Build Options

Development install (editable, source in current directory):

pip install --extra-index-url=https://download.pytorch.org/whl/xpu -e . -v
# Faster: skip build isolation if dependencies are already present
pip install --no-build-isolation -e . -v

Standard install (to site-packages):

pip install --extra-index-url=https://download.pytorch.org/whl/xpu .
# or
pip install --no-build-isolation .

Build a wheel (output goes to dist/):

pip wheel --extra-index-url=https://download.pytorch.org/whl/xpu --wheel-dir dist .
# or
pip wheel --no-build-isolation --wheel-dir dist .

Incremental rebuild (fastest for iterative development):

python -m build --wheel --no-isolation

Using with vLLM

After vLLM RFC#33214 was completed, vLLM-XPU migrated to a vllm-xpu-kernels-based implementation. Installing the latest vLLM for XPU will pull in vllm-xpu-kernels automatically as a wheel dependency — no manual integration is required.

Kernel Configuration

By default, vLLM-XPU compiles kernels for common models (Llama, Qwen, DeepSeek). For customization:

VLLM_CHUNK_PREFILL_CONFIG=chunk_prefill_full.conf VLLM_PAGED_DECODE_CONFIG=paged_decode_full.conf pip install .

See KERNEL_CONFIGURATION.md for detailed guidance on kernel configuration, presets, and troubleshooting missing kernels.

Testing

Run the full test suite with pytest:

pytest tests/

Individual test modules cover activations, cache operations, attention, MoE, LoRA, quantization, and memory utilities. See the tests/ directory for the complete list.

Benchmarks

Benchmark scripts for individual kernels are in the benchmark/ directory:

python benchmark/benchmark_layernorm.py
python benchmark/benchmark_lora.py
python benchmark/benchmark_grouped_topk.py
# etc.

License

This project is licensed under the Apache License 2.0. See the LICENSE file for details.