Skip to main content

qnq's not quantization

Project description

QNQ -- QNQ's not quantization

version 1.1.0 2021.2.5

Description

The toolkit is for Techart algorithm team to quantize their custom neural network's pretrained model. The toolkit is beta now, you can contact me with email(dongz.cn@outlook.com) for adding ops and fixing bugs.

How to install

pip install qnq

How to use

This README.MD is in very early stages, and will be updated soon. you can visit https://git.zwdong.com/zhiwei.dong/qnq_tutorial for more examples for QNQ.

  1. Prepare your model.

    1. Check if your model contains non-class operator, like torch.matmul.
    2. If True, add from qnq.operators.torchfunc_ops import * to your code.
    3. Then use class replace non-class operator, you can refer fellow #! add by dongz
    class BasicBlock(nn.Module):
        expansion = 1
    
        def __init__(self, inplanes, planes, stride=1, downsample=None):
            super(BasicBlock, self).__init__()
            self.conv1 = conv3x3(inplanes, planes, stride)
            self.bn1 = nn.BatchNorm2d(planes)
            self.relu1 = nn.ReLU(inplace=True)
            self.relu2 = nn.ReLU(inplace=True)
            self.conv2 = conv3x3(planes, planes)
            self.bn2 = nn.BatchNorm2d(planes)
            self.downsample = downsample
            self.stride = stride
    
            #! add by dongz
            self.torch_add = TorchAdd()
    
        def forward(self, x):
            identity = x
    
            out = self.conv1(x)
            out = self.bn1(out)
            out = self.relu1(out)
    
            out = self.conv2(out)
            out = self.bn2(out)
    
            if self.downsample is not None:
                identity = self.downsample(x)
    
            #! add by dongz
            out = self.torch_add(out, identity)
            # out += identity
            out = self.relu2(out)
    
            return out
    
  2. Prepare 'metrics', 'metrics_light'(optional) and 'steper'.

    1. Choose at least 1k data to calibration your quantized model.
    2. 'metrics' inference without input params, return metrics value(a float number).
    3. 'metrics_light' inference without input params, return metrics value(a float number), you can choose 1/10 testsets to test.
    4. 'steper' done inference and without input params too, but add quant.step(), and no return.
    5. Check qnq_tutorial for details.
  3. Prepare pretrained checkpoints.

    1. Train your model and use torch.save() to save your checkpoints.
    2. Use checkpoints = torch.load(checkpoints_path) and model.load_state_dict(checkpoints) to load your checkpoints.
  4. Quantize

    1. For code
      1. Add from qnq import QNQ
      2. Add quant = QNQ(model, save_path, config_path, metrics, metrics_light, steper).
      3. Add quant.search()
    2. First run the program will exit, but the config_path will show a yaml file.
    3. Edit config.yaml and rerun for quantization.

Operators supported

  • Convolution Layers
    • Conv
    • ConvTranspose
  • Pooling Layers
    • MaxPool
    • AveragePool
    • AdaptiveAvgPool
  • Activation
    • Relu、Relu6
    • PRelu、LeakyRelu
    • LogSoftmax
  • Normalization Layers
    • BatchNorm
    • LayerNorm
  • Recurrent
    • LSTM
  • Linear Layers
    • Linear
  • Vision Layers
    • Upsample
    • Embedding
  • Torch Function
    • Add, Sum, Minus, DotMul, MatMul, Div,
    • Sqrt, Exp
    • Sin, Cos
    • SoftMax, Sigmoid, Tanh
    • TorchTemplate, TorchDummy

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

qnq-1.1.2.tar.gz (39.9 kB view details)

Uploaded Source

Built Distribution

qnq-1.1.2-py2.py3-none-any.whl (46.7 kB view details)

Uploaded Python 2Python 3

File details

Details for the file qnq-1.1.2.tar.gz.

File metadata

  • Download URL: qnq-1.1.2.tar.gz
  • Upload date:
  • Size: 39.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.6.13

File hashes

Hashes for qnq-1.1.2.tar.gz
Algorithm Hash digest
SHA256 4a8c7f56155a71dd865d651a3ded597ec737563532827a135efef32565fa1b6a
MD5 741d2ebe3b75812c3b82663a903c976e
BLAKE2b-256 0f4a0d1cf687d7830e09f437f62ea2ed080253aec308a42819df35efa4873ed6

See more details on using hashes here.

File details

Details for the file qnq-1.1.2-py2.py3-none-any.whl.

File metadata

  • Download URL: qnq-1.1.2-py2.py3-none-any.whl
  • Upload date:
  • Size: 46.7 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.6.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.1 CPython/3.6.13

File hashes

Hashes for qnq-1.1.2-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 92b87402a8ae3ab0966c75a88d45e5b6d0a83a6561cc7fc73e5a3c1da4ce0214
MD5 c23497486c28b1007f0b089c420d5c2b
BLAKE2b-256 4fa37009fd9b29383cced2bf7db4cb9f7b7e25b151e8320ca165fa2c8ecb4e19

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page