STIR -- shared invariance measure
Project description
Similarity Throught Inverted Representations (STIR) Paper
Measuring Representational Robustness of Neural Networks Through Shared Invariances, ICML 2022. Vedant Nanda, Till Speicher, Camila Kolling, John P. Dickerson, Krishna P. Gummadi, Adrian Weller
Installation
pip install stir-invariance
Or you can install it from source. First clone this repo and then run
python setup.py install
Quick Start
import stir
import stir.model.tools.helpers as helpers
import stir.helper as hp
test_dataloader = ... ## (instance of torch.utils.data.DataLoader,
## should return (images, labels) in each iter)
## images should *not* be normalized since
## normalization is performed in model's forward pass
model1 = ... ## instance of torch.nn.Module
model1_dataset = 'cifar10'
normalizer1 = helpers.InputNormalize(*hp.DATASET_TO_MEAN_STD[model1_dataset])
## or any instance of torch.nn.Module that performs input normalization
model2 = ... ## instance of torch.nn.Module
model2_dataset = 'cifar10'
normalizer2 = helpers.InputNormalize(*hp.DATASET_TO_MEAN_STD[model2_dataset])
## or any instance of torch.nn.Module that performs input normalization
total_images = 1000 # number of images to use for computing STIR
# computes STIR between penultimate layer of model1 and model2
stir_score = stir.STIR(model1, model2,
normalizer1, normalizer2,
(test_dataloader, total_images))
stir_score.m1m2 ## STIR(m1|m2)
stir_score.m2m1 ## STIR(m2|m1)
stir_score.rsm ## Underlying similarity measure (default: Linear CKA)
Citation
If you find our work useful, please cite it:
@inproceedings{nanda2022measuring,
title={Measuring Representational Robustness of Neural Networks Through Shared Invariances},
author={Nanda, Vedant and Speicher, Till and Kolling, Camilla and Dickerson, John P. and Gummadi, Krishna P. and Weller, Adrian},
booktitle={ICML},
year={2022}
}
Acknowledgements
This repo borrows code from the robustness library to invert representations (and train models). The minibatch CKA implementation is inspired by PyTorch-Model-Compare and full batch from @yuanli2333's CKA implementation. Models for TRADES and MART were trained using the repos made public by authors of the respective papers.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for stir_invariance-0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 687912c6a1a705162d9d434948656ad76f4a953f40bdf1bd706b389b21ede96a |
|
MD5 | 7436d25759f9ea9b69f3276d6bd40316 |
|
BLAKE2b-256 | 83ebee2f6b4c21c74c61b1295abe0892eb20d022c3bcc8661465daf10b97206d |