AutoAWQ Kernels implements the AWQ kernels.
Project description
AutoAWQ Kernels
AutoAWQ Kernels is a new package that is split up from the main repository in order to avoid compilation times.
Requirements
-
Windows: Must use WSL2.
-
NVIDIA:
- GPU: Must be compute capability 7.5 or higher.
- CUDA Toolkit: Must be 11.8 or higher.
-
AMD:
- ROCm: Must be 5.6 or higher. Build from source
Install
Install from PyPi
The package is available on PyPi with CUDA 12.4.1 wheels:
pip install autoawq-kernels
Build from source
To build the kernels from source, you first need to setup an environment containing the necessary dependencies.
Build Requirements
- Python>=3.8.0
- Numpy
- Wheel
- PyTorch
- ROCm: You need to install the following packages
rocsparse-dev hipsparse-dev rocthrust-dev rocblas-dev hipblas-dev
.
Building process
pip install git+https://github.com/casper-hansen/AutoAWQ_kernels.git
Notes on environment variables:
TORCH_VERSION
: By default, we build using the current version of torch bytorch.__version__
. You can override it withTORCH_VERSION
.CUDA_VERSION
orROCM_VERSION
can also be used to build for a specific version of CUDA or ROCm.
CC
andCXX
: You can specify which build system to use for the C code, e.g.CC=g++-13 CXX=g++-13 pip install -e .
COMPUTE_CAPABILITIES
: You can specify specific compute capabilities to compile for:COMPUTE_CAPABILITIES="75,80,86,87,89,90" pip install -e .
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
File details
Details for the file autoawq_kernels-0.0.9-cp312-cp312-win_amd64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp312-cp312-win_amd64.whl
- Upload date:
- Size: 2.8 MB
- Tags: CPython 3.12, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | f259e7c60b11fa0689bb337dd4456319787256cbd2a8e4a491f01b51bb6c43d1 |
|
MD5 | 9837d7e837a0349592b8ecac37c8f674 |
|
BLAKE2b-256 | dca32a1966f685a980c1ad5662f92a1aa1a84608fb376b534301330487fc1db8 |
File details
Details for the file autoawq_kernels-0.0.9-cp312-cp312-manylinux2014_x86_64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp312-cp312-manylinux2014_x86_64.whl
- Upload date:
- Size: 37.4 MB
- Tags: CPython 3.12
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4c41a71af1d5a75e52c9833b9c48237b04d3b0eee26d712fc1b074af9135afc8 |
|
MD5 | f5252fdc2802b899d3f13dfdf7634faf |
|
BLAKE2b-256 | 4aef613310287e009d35b740b76b96913ea87e5522d7f91845bf817d4ed0abd9 |
File details
Details for the file autoawq_kernels-0.0.9-cp311-cp311-win_amd64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp311-cp311-win_amd64.whl
- Upload date:
- Size: 2.8 MB
- Tags: CPython 3.11, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8c7f2404b3aa448ff77872dd6ba2963ce8b685d8aa73ef65fd1b8bc85d92b17d |
|
MD5 | c7d056e76910506db6113eb93875dbed |
|
BLAKE2b-256 | 8658093d5cd5d48f82aeb29ab0dcf76c3ed1a636c2c249ab517773190cb77a67 |
File details
Details for the file autoawq_kernels-0.0.9-cp311-cp311-manylinux2014_x86_64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp311-cp311-manylinux2014_x86_64.whl
- Upload date:
- Size: 37.4 MB
- Tags: CPython 3.11
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | fe800a6538691afaa77abe7c8b2b0a121351843f048d54e11d617d604dcba48f |
|
MD5 | 896e8f8891837ddc5681ec1e6e631034 |
|
BLAKE2b-256 | 4eb0e9f7142e58e5539892cb0558e9a24d894f095a9904b4014f892a43c39229 |
File details
Details for the file autoawq_kernels-0.0.9-cp310-cp310-win_amd64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp310-cp310-win_amd64.whl
- Upload date:
- Size: 2.7 MB
- Tags: CPython 3.10, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | cd7d3db501068b3a12094a07921d985a57e640725cdda1252d4b135ed6aeaa65 |
|
MD5 | c5aa60c9ea4f650e21f75f532dfdf619 |
|
BLAKE2b-256 | 2e35471f80543e31aef5097b037cc9626a47a0edb20311c3951060673cc83868 |
File details
Details for the file autoawq_kernels-0.0.9-cp310-cp310-manylinux2014_x86_64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp310-cp310-manylinux2014_x86_64.whl
- Upload date:
- Size: 37.4 MB
- Tags: CPython 3.10
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ed8f4d744df21beae445efb1de54061bffc5fccbfefc8ae65c1dc10d08f90052 |
|
MD5 | 667f6d43ceb618a0a8f0a77f07ad8b83 |
|
BLAKE2b-256 | 98a6c48cf823c2d29731ae262a05e17d317165df7bef68a486e5840405b70cc0 |
File details
Details for the file autoawq_kernels-0.0.9-cp39-cp39-win_amd64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp39-cp39-win_amd64.whl
- Upload date:
- Size: 2.7 MB
- Tags: CPython 3.9, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6ad12dd68b0932182678f2f9fbee87452707b81f0e8dad242d23af018358f030 |
|
MD5 | f45a15e6ebd21cd136d2857e3e5115f8 |
|
BLAKE2b-256 | 3b6062b1ff1406dd4fdd90ad8dce840f8e4c19f01b755cad86d6498e55dfa373 |
File details
Details for the file autoawq_kernels-0.0.9-cp39-cp39-manylinux2014_x86_64.whl
.
File metadata
- Download URL: autoawq_kernels-0.0.9-cp39-cp39-manylinux2014_x86_64.whl
- Upload date:
- Size: 37.3 MB
- Tags: CPython 3.9
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | b6baf039c22deb02f2ae194fdd77551b3c85c8f8a77b749f7caa17dacf986adb |
|
MD5 | 759cd2a1d37a14f2c999b34481fbd2ff |
|
BLAKE2b-256 | 45bb4554a1b3cf0d29acf8ef449e875ae24c6935d68858876b0ec44a2fd4c2e5 |