structured-pruning-adapters 0.5.2

Structured Pruning Adapters for PyTorch

Structured Pruning Adapters for PyTorch

pip install structured-pruning-adapters

A happy mariage 👰‍♀️🤵‍♂️

Pruning is an effective method for reducing the size of neural networks. Besides reducing the parameter count, the process can accelerate inference as well. CPUs can handle sparse weights just fine, but GPUs need structured weights for an acceleration to happen. A structured approach to pruning i.e., removing network channels [paper] or blocks of weights [paper], generally yields speedups as well

+

Adapters [paper] have emerged as an alternative to fine-tuning, in which the prior network weights are unaltered, and a new set of adapters weights are added to the network to learn a specific task. Some types of adapters add new layers, others are fusible with existing weights and don't have a run-time overhead. When a single base-model is deployed with many specialised models, these structures can save a lot of parameters compared with full fine-tuning.

=

Structured Pruning Adapters are the offspring of Structured Pruning and Fusible Adapters, and can be used for Transfer Learning which has:

• ✅ Extremely few learned parameters (binary pruning mask + masked adapter weights) 👌
• ✅ Accelerated network inference 🏎💨

How to use this library

Use in conjunction with any Structured Pruning technique.

1. Install the library:

   pip install structured-pruning-adapters
   

2. Replace Linear and Conv layers with an SP Adapter:

   from torch.nn import Linear
   from sp_adapter import SPLoRA
   
   reg_lin = Linear(256, 512, bias=True)
   spa_lin = SPLoRA(reg_lin, rank=32)
   
   # Or replace all applicable layers in a network
   spa_net = SPLoRA(reg_net, rank=32)
   

3. Employ any Structured Pruning method, e.g this or that.

4. Get pruned SP Adapter weights:

   # Specify mask
   spa_lin.configure_parameter_read(
       adapter_weights_only=True,
       in_features_mask=torch.tensor([1, 0, ..., 1], dtype=torch.bool)
       out_features_mask=torch.tensor([0, 1, ..., 1], dtype=torch.bool),
   )   # 👆 masks are learned via your choice of Structured Pruning method
   
   # Read parameters as usual
   spa_lin.parameters()
   spa_lin.named_parameters()
   spa_lin.state_dict()
   

Demo

See also notebooks/demo.ipynb for a hands-on demo.

Structured Pruning Low-Rank Adapter (SPLoRA) for Channel Pruning

from sp_adapters import SPLoRA

Adds a low-rank bottle-neck projection in projection in parallel with the main weights projection.

Structured Pruning Low-rank PHM Adapter (SPLoPA) for Block Pruning

from sp_adapters import SPLoPA

Uses a variation on the Parameterized Hypercomplex Multiplication (PHM) layer [paper] with shared low-rank prototypes for block-sparse adaptation.

Citation

If you enjoy this work, please consider citing it

@article{hedegaard2022cotrans,
  title={Structured Pruning Adapters},
  author={Lukas Hedegaard, Aman Alok, Juby Jose, Alexandros Iosifidis},
  journal={preprint, arXiv:TBD},
  year={2022}
}

Acknowledgement

This work was done in conjunction with a research exchange at Cactus Communications 🌵.

This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR) 🇪🇺.