An implementation of methods described in Git Re-basin-paper by Ainsworth et al.
Project description
rebasin
An implementation of methods described in "Git Re-basin"-paper by Ainsworth et al.
Installation
pip install rebasin
Usage
Currently, only weight-matching is implemented as a method for rebasing, and only a simplified form of linear interpolation is implemented.
from rebasin import PermutationCoordinateDescent
from rebasin import interpolation
model_a, model_b, train_dl, val_dl, loss_fn = ...
input_data = next(iter(train_dl))[0]
# Rebasin
pcd = PermutationCoordinateDescent(model_a, model_b, input_data)
pcd.calculate_permutations()
pcd.apply_permutations()
# Interpolate
lerp = interpolation.LerpSimple(
models=[model_a, model_b],
loss_fn=loss_fn,
train_dataloader=train_dl,
val_dataloader=val_dl
)
lerp.interpolate(steps=10)
# Access model with lowest validation loss:
lerp.best_model
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
rebasin-0.0.15.tar.gz
(17.6 kB
view hashes)
Built Distribution
rebasin-0.0.15-py3-none-any.whl
(18.7 kB
view hashes)