Last released Mar 21, 2024
A general x-bit quantization engine for LLMs,[2-8] bits, awq/gptq/hqq
Supported by