Conch :shell:

A "standard library" of Triton kernels.

What is Conch?

Conch is a central repository of Triton kernels for accelerating common AI operations. We strive to provide performant, well-written kernels that can be easily integrated into other projects. We also strive to support multiple hardware platforms (currently Nvidia and AMD).

Key Features

We support each of the following operations. Each operation is complete with a PyTorch-only reference implementation (and sometimes a reference implementation provided by another library, like vLLM), a microbenchmark, and a unit test.

• Activation functions
  • GeLU and mul
  • SiLU and mul
• Attention
  • Paged Attention (Flash-Decoding with Paged KV Cache)
  • Varlen Attention (Prefill/decode attention with paged KV cache)
• Embedding
  • Rotary embedding
• Normalization
  • Gemma-style RMS norm
  • Llama-style RMS norm
• Quantization
  • bitsandbytes
    • NF4/FP4/8-bit blockwise quantize/dequantize
  • FP8 static quantization
  • Int8 static quantization
  • GEMM
    • Mixed-precision
    • Scaled
• Vision
  • BEVPool
  • Non-Max Suppression (NMS)
  • Voxelization
• vLLM
  • KV cache operations
    • Copy blocks
    • Reshape and cache

Performance

The goal of Conch is not to claim that our operations are faster than CUDA implementations. Our goal is to write Triton operations that are as fast as the state-of-the-art CUDA implementations. This allows developers on any hardware platform (Nvidia, AMD, etc.) access to the same, performant kernels.

Below is a table comparing the relative performance of our Triton kernels to CUDA baselines (on NVIDIA H100). The listed runtime is the median runtime from 10,000 iterations on our microbenchmarks. Note: it's difficult to express the performance of a kernel with a single number (performance will vary with input sizes, data types, etc.). We tried our best to choose representative parameters for a fair comparison. Most relevant parameters are specified via CLI parameters to the microbenchmarks (benchmarks/), so feel free to collect your own results based on your use case. CUDA runtimes collected via vLLM and bitsandbytes (vllm==0.9.1 and bitsandbytes==0.46.0).

For additional analysis of kernel performance, check out our performance docs.

Supported platforms

Supported platforms:

• Nvidia A10, CUDA 12.2
• Nvidia H100, CUDA 12.2
• AMD MI300X, ROCm 6.2.4

Work-in-progress platforms:

• XPU
• CPU

Getting Started

Users

Check out the installation instructions to get started!

Developers

Check out the developer instructions to get started!

Open-source credits

We were inspired by and leverage components of the following libraries:

• BEVFusion
• bitsandbytes
• GemLite
• Torchvision
• vLLM

License

Copyright 2025 Stack AV Co. Licensed under the Apache License, Version 2.0.