vinn 1.1

Visual Interpretation of Neural Networks

VINN

VINN is Visual Interpretation of Neural Networks

VINN includes a series of interpretability algorithms, like LIME, Grad CAM, Integrated Gradient, saliency maps and so on.

VINN is a graduation project, and will update to new version if possible.

Target audience

Developers, who are looking to understand what their deep models have learnt, or researchers, who want to test their new algorithms are the potential audiences for VINN.

VINN standardize the algorithm framework, so that you can follow the framework to construct your own algorithm and test it.

Installation

requirements

you can create a new environment by

 conda create -f requirements

or make sure your own environments have

• python >= 3.6

• pytorch >= 1.2

installing the latest release

pip:

pip install vinn

conda:

conda install vinn

or you can download from GitHub and install in the repository

https://github.com/Torato-Taraka/VINN.git
pip install vinn-1.0-py3-none-any.whl

Getting start

import torch
import torchvision.transforms as tfs
from torchvision.models import resnet18
import cv2

model = resnet18(pretrained=True)
transform = tfs.Compose([
    tfs.ToTensor(),
    tfs.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
last_conv = model.layer4[-1]

image_path = 'test.jpg'
image = cv2.imread(image_path)
image = cv2.resize(image, (256, 256))
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    
gpu = True

Till now, necessary parameters(model, image, transform, gpu) is ready, and you can use VINN to interpret your model

import vinn
gradcam = vinn.GradCAM(model=model,
                      image=image,
                      last_conv=last_conv,
                      transform=transform,
                      gpu=gpu)
gradcam.forward()

After forward, the algorithm finish processing, and the result is stored in gradcam.mask.

You can visualize the result like the following:

params = {
    "original image", gradcam.image,
    "GradCAM", gradcam.mask,
    "masked image", gradcam.masked_image
}
vinn.subplot([params])

Note that you should pass a list of dicts, since that you may need to interpret several images at the same time.

And you can see how the image masked by calling gradcam.masked_image.

But pay attention that NOT all the algorithms generates mask and original image together, only GradCAM, GuidedGradCAM, LIME and Sensitivity have masked_image.

Of course you can generate by yourself and maybe the later version will achieve the function.

Customize models, datasets and algorithms

VINN support customizing your model, datasets and even algorithms

models

You should organize your model like

from vinn.model import Models

@Models("yourmodel")
def yourmodel():
    class yourmodel(nn.Module):
        def __init__(self):
        	...
    return yourmodel()

This is for registering your algorithm in order to management

datasets

Similar to customize models

from vinn.datasets import DataSet

@DataSet("yourdataset")
def yourdataset():
    class yourdataset:
        def __init__(self):
            ...
            
        def __getitem__(self, index):
            image = self.transform(self.images[index])
            lable = self.lables[index]
            return image, lable
        
        def __len__(self):
            return len(self.labels)
    return yourdataset()

__getitem__() and __len__() is necessary, you can see the details at [torch.utils.data.Dataset](torch.utils.data — PyTorch 1.10 documentation)

And you can see all the existing dataset by

from vinn import list_dataset
list_dataset()

train model

from vinn import trainer
from torchvision import transforms as tfs
transform = tfs.Compose([
    ...
])
trainer(
    model_name='yourmodel',
    dataset_name='yourdataset',
    classes=...,
    transform=transform,
    train_size=...,
    test_size=...,
    lr=...,
    epochs=...,
    gpu=True
)

• model_name: the string in the @Models() you define
• dataset_name: the string in the @Dataset() you define
• classes: the classes of your dataset
• transform: you define it
• train_size: the size of a sample from your train data
• test_size: the size of a sample from your test data
• lr: learning rate
• epochs: training times
• gpu: whether use cuda or not

algorithms

You should first organize your algorithm like

from vinn.algorithm import Algorithm 
class youralgo(Algorithm):
    def __init__(self, model, image, transform, gpu, ...):
        super().__init__(model, image, transform, gpu)
        ...
        
    def forward(self):
        ...
        self.mask = mask

Algorithm is the father class of all algorithms

The most important thing is that you must pass the result to self.mask, so that you can easily visualize your result.

Or you can return your result, but I don't think it good for your following operation.

References of algorithms

• IntegratedGradients: Axiomatic Attribution for Deep Networks, Mukund Sundararajan et al. 2017

• SmoothGrad: SmoothGrad: removing noise by adding noise, Daniel Smilkov et al. 2017

• SaliencyMap : Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps, K. Simonyan, et. al. 2014

• GradCAM, GuidedGradCAM: Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, Ramprasaath R. Selvaraju et al. 2017

• LIME:"Why Should I Trust You?": Explaining the Predictions of Any Classifier

• GBP(Guided Backpropagation):Striving for Simplicity: The All Convolutional Net, Jost Tobias Springenberg et al. 2015

• Sensitivity:Interpretable Explanations of Black Boxes by Meaningful Perturbation

License

VINN is MIT licensed, as found in LICENSE file.