amd-quark 0.10

AMD Quark is a comprehensive cross-platform toolkit designed to simplify and enhance the quantization of deep learning models. Supporting both PyTorch and ONNX models, AMD Quark empowers developers to optimize their models for deployment on a wide range of hardware backends, achieving significant performance gains without compromising accuracy.

AMD Quark Model Optimizer

PyTorch Examples | ONNX Examples | Documentation | Release Notes

AMD Quark is a comprehensive cross-platform toolkit designed to simplify and enhance the quantization of deep learning models. Supporting both PyTorch and ONNX models, AMD Quark empowers developers to optimize their models for deployment on a wide range of hardware backends, achieving significant performance gains without compromising accuracy.

Features

Feature Set	PyTorch backend	ONNX backend
Data Types	int4, uint4, int8, uint8, float16, bfloat16, OCP FP8 E4M3/E5M2, OCP MX INT8, OCP MX FP4, OCP MX FP6 E3M2/E2M3, OCP MX FP8 E4M3/E5M2	int4, uint4, int8, uint8, int16, uint16, int32, uint32, float16, bfloat16, BFP16, MX4/MX6/MX9, OCP MX INT8, OCP MX FP4, OCP MX FP6 E3M2/E2M3, OCP MX FP8 E4M3/E5M2
Quant Mode	eager mode, FX graph mode	ONNX graph mode
Quant Strategy	static quant, dynamic quant, weight-only	static quant, dynamic quant, weight-only
Quant Scheme	per-tensor, per-channel, per-group	per-tensor, per-channel
Symmetric	symmetric, asymmetric	symmetric, asymmetric
Calibration Method	MinMax, Percentile, MSE	MinMax, Percentile, MinMSE, Entropy, NonOverflow
Scale Type	float16, float32	float16, float32
KV-Cache Quant	FP8 KV-Cache Quant	N/A
Supported Ops.	nn.Linear, nn.Conv2d, nn.ConvTranspose2d, nn.Embedding, nn.EmbeddingBag,	Almost all ONNX ops,
	nn.BatchNorm2d, nn.BatchNorm3d, nn.LeakyReLU, nn.AvgPool2d, nn.AdaptiveAvgPool2d	see Full List
Pre-Quant Optimization	SmoothQuant	QuaRot, SmoothQuant, CLE
Quantization Algorithm	AWQ, GPTQ, Qronos	AdaQuant, AdaRound, GPTQ, Bias Correction
Export Format	ONNX, JSON-Safetensors, GGUF(Q4_1)	N/A
Operating Systems	Linux {ROCm, CUDA, CPU}, Windows {CPU}	Linux {ROCm, CUDA, CPU}, Windows {CUDA, CPU}

Model Support Table

Quantization Technique	Supported Models
LLM Pruning	Model Support
LLM Post Training Quantization (PTQ)	Model Support
LLM Quantization Aware Training (QAT)	Model Support
Vision Model Quantization	Model Support
Quark for ONNX	Model Support

Installation

Official releases of AMD Quark are available on PyPI https://pypi.org/project/amd-quark/, and can be installed with pip:

pip install amd-quark

[!NOTE]
For full instructions to install AMD Quark from Python wheels or ZIP files, refer to our 🛠️Installation Guide. The Installation Guide also contains verification steps that apply to building from source.

Installing from Source

1. Clone or download this repository.
2. Follow the steps from the PyTorch website to install the appropriate PyTorch package for your system.
3. You can then build and install AMD Quark, and its dependencies, which are detailed in requirements.txt, by running:

git clone --recursive https://github.com/AMD/Quark
cd Quark

# [Optional] run git submodule if you are updating an existing Quark repository
git submodule sync
git submodule update --init --recursive

pip install .

Resources

AMD Quark's documentation site contains Getting Started, API documentation for both PyTorch and ONNX backends, and other detailed information. The Installation Guide includes our Recommended First Time User Installation guide, to get set up with Quark quickly. Check out our Frequently Asked Questions for both PyTorch and ONNX for more details.

• 📖Documentation
• 📄FAQ (PyTorch)
• 📄FAQ (ONNX)

AMD Quark provides examples of Language Model and Image Classification model quantization, which can be found under examples/torch/ and examples/onnx/. These examples are documented here:

• 💡PyTorch Examples
• 💡ONNX Examples

The examples folder also contain integrations of other quantizers under examples/torch/extensions/. You can read about those here:

• Brevitas Integration
• Integration with AMD Pytorch-light (APL).

Contributing

AMD Quark is not set up to accept community contributions (bug reports, feature requests, or Pull Requests) just yet. Please watch this space!

License and Copyright

Copyright (C) 2025, Advanced Micro Devices, Inc. All rights reserved. SPDX-License-Identifier: MIT. See LICENSE file for detail.