kernels
The Kernel Hub allows Python libraries and applications to load compute kernels directly from the Hub. To support this kind of dynamic loading, Hub kernels differ from traditional Python kernel packages in that they are made to be:
- Portable: a kernel can be loaded from paths outside
PYTHONPATH. - Unique: multiple versions of the same kernel can be loaded in the same Python process.
- Compatible: kernels must support all recent versions of Python and the different PyTorch build configurations (various CUDA versions and C++ ABIs). Furthermore, older C library versions must be supported.
Usage
Kernels depends on torch>=2.5 and CUDA for now.
Here is how you would use the activation kernels from the Hugging Face Hub:
import torch
from kernels import get_kernel
# Download optimized kernels from the Hugging Face hub
activation = get_kernel("kernels-community/activation")
# Random tensor
x = torch.randn((10, 10), dtype=torch.float16, device="cuda")
# Run the kernel
y = torch.empty_like(x)
activation.gelu_fast(y, x)
print(y)
These kernels can be built from the kernel-builder library.
If you're looking to better understand how these kernels are structured, or looking to build your own kernels, please take a look at the following guide: writing kernels.
Installation
To install kernels, we recommend installing from the pypi package:
pip install kernels
You should then be able to run the script above (also in examples/basic.py):
python examples/basic.py
Docker Reference
build and run the reference examples/basic.py in a Docker container with the following commands:
docker build --platform linux/amd64 -t kernels-reference -f docker/Dockerfile.reference .
docker run --gpus all -it --rm -e HF_TOKEN=$HF_TOKEN kernels-reference
Locking kernel versions
Projects that use setuptools can lock the kernel versions that should be
used. First specify the accepted versions in pyproject.toml and make
sure that kernels is a build dependency:
[build-system]
requires = ["kernels", "setuptools"]
build-backend = "setuptools.build_meta"
[tool.kernels.dependencies]
"kernels-community/activation" = ">=0.0.1"
Then run kernel lock . in the project directory. This generates a kernels.lock file with
the locked revisions. The locked revision will be used when loading a kernel with
get_locked_kernel:
from kernels import get_locked_kernel
activation = get_locked_kernel("kernels-community/activation")
Note: the lock file is included in the package metadata, so it will only be visible
to kernels after doing an (editable or regular) installation of your project.
Pre-downloading locked kernels
Locked kernels can be pre-downloaded by running kernel download . in your
project directory. This will download the kernels to your local Hugging Face
Hub cache.
The pre-downloaded kernels are used by the get_locked_kernel function.
get_locked_kernel will download a kernel when it is not pre-downloaded. If you
want kernel loading to error when a kernel is not pre-downloaded, you can use
the load_kernel function instead:
from kernels import load_kernel
activation = load_kernel("kernels-community/activation")
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file kernels-0.1.7.tar.gz.
File metadata
- Download URL: kernels-0.1.7.tar.gz
- Upload date:
- Size: 11.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.1.0 CPython/3.13.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0b5f90dd834c9a9174e2907db58c960838c7c6dc8fa905ef84a96ee3f35288ce
|
|
| MD5 |
e7e900e512747dc75a301de5fc27e528
|
|
| BLAKE2b-256 |
5a9c5d0b05e4b1d46a62b451a24a9b463520e148b318a97d2d6e77a3f53f9c05
|
File details
Details for the file kernels-0.1.7-py3-none-any.whl.
File metadata
- Download URL: kernels-0.1.7-py3-none-any.whl
- Upload date:
- Size: 10.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.1.0 CPython/3.13.2
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f840d4fee6ac53a2c6d6df20748401b46656e9dd2807472d4c6436ea0ae1ec74
|
|
| MD5 |
a7073d04f7593591016d0b4053bc6776
|
|
| BLAKE2b-256 |
c636245f5f4c41da3d87b34da63980add2b42f8b02e6e7b4e31385c6e9fb038a
|
