Tool for inter-architecture weight transfer
Project description
Inter-Architecture Knowledge Transfer
iatransfer is a PyTorch package for transferring pretrained weights between models of different architectures instantaneously.
Drastically speed up your training process using two additional lines of code.
Installation
pip install iatransfer
Usage
- simple
import torch
from iatransfer.toolkit import IAT
transfer = IAT()
# run training on Model1()
model_from: nn.Module = Model1()
train(model_from)
# instantiate new model
model_to: nn.Module = Model2()
# enjoy high-accuracy initialization
transfer(model_from, model_to)
- parametrization
from iatransfer.toolkit import IAT
iat = IAT(standardization='blocks', matching='dp', score='autoencoder', transfer='trace')
# ==== or
iat = IAT(matching=('dp', {'param': 'value'}))
# ==== or
from iatransfer.toolkit.matching.dp_matching import DPMatching
iat = IAT(matching=DPMatching())
- plugins
from iatransfer.toolkit.base_matching import Matching
class CustomMatching(Matching):
def match(self, from_module, to_module, *args, **kwargs)
# provide your implementation
# This will instantiate the above CustomMatching in IAT
iat = IAT(matching='custom')
Citation
When referring to or using iatransfer in a scientific publication, please consider including citation to the following thesis:
@manual{
iat2021,
title = {Inter-Architecture Knowledge Transfer},
author = {Maciej A. Czyzewski and Daniel Nowak and Kamil Piechowiak},
note = {Transfer learning between different architectures},
organization = {Poznan University of Technology},
type = {Bachelor’s Thesis},
address = {Poznan, Poland},
year = {2021}
}
Development
Init:
./dev/init.sh
Run tests:
nosetests tests
Install in edit mode:
pip install -e .
Research reproduction:
Copy the source code to the GCP cloudshell or install iatransfer_research
package.
Run:
/bin/bash ./scripts/research/iatransfer_full_run.sh
or
iatransfer_full_run.sh
if iatransfer_research
has been installed.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for iatransfer_research-1.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eebc498965a796391da91f653614cbc7cead4a8276208e7c5eb2ee2c8440313e |
|
MD5 | 72e4135de8611fdf6d6608c9582879ac |
|
BLAKE2b-256 | e20252a9df49ac8f294abdd216b094303e2babf83e728c9a762aef9b0093a353 |