Skip to main content

Visualization toolkit for neural networks in PyTorch

Project description

FlashTorch

PyPI - Python Version PyPI PyPI - License DOI

Visualizaion toolkit implemented in PyTorch for inspecting what neural networks learn in image recognition tasks (feature visualizaion).

The project is very much work in progress, and I would appreciate your feedback!

It currently supports visualizaion of saliency maps for all the models available under torchvision.models.

Overview

Installation

If you are installing flashtorch for the first time:

$ pip install flashtorch

Or if you are upgrading it:

$ pip install flashtorch -U

API guide

An API guide is under construction, so this is a temporary workaround.

These are currently available modules.

  • flashtorch.utils: some useful utility functions for data handling & transformation
  • flashtorch.utils.imagenet: ImageNetIndex class for easy-ish retrieval of class index
  • flashtorch.saliency.backprop: Backprop class for calculating gradients
  • flashtorch.activmax.gradient_ascent: GradientAscent class for activation maximization

You can inspect each module with Python built-in function help. The output of that is available on Quick API Guide for your convenience.

How to use

Here are some handy notebooks showing examples of using flashtorch.

Image handling

Notebook: Image handling

Saliency maps

Saliency in human visual perception is a subjective quality that makes certain things within the field of view stand out from the rest and grabs our attention.

Saliency maps in computer vision provide indications of the most salient regions within images. By creating a saliency map for neural networks, we can gain some intuition on "where the network is paying the most attention to" in an imput image.

Using flashtorch.saliency module, let's visualise image-specific class saliency maps of AlexNet pre-trained on ImageNet classification tasks.

Great gray owl (class index 24): The network is focusing on the sunken eyes and the round head for this owl.

Saliency map of great grey owl in Alexnet

Refer to the notebooks below for more examples:

Activation maximization

Activation maximization is one form of feature visualization that allows us to visualize what CNN filters are "looking for", by applying each filter to an input image and updating the input image so as to maximize the activation of the filter of interest (i.e. treating it as a gradient ascent task with filter activation values as the loss).

Using flashtorch.activmax module, let's visualise images optimized with filters from VGG16 pre-trained on ImageNet classification tasks.

Activation maximization of conv5_1 filters from VGG16

We can see that, in the earlier layers (conv1_2, conv2_1), filters get activated by colors and simple patterns such as virtical, horisontal and diagonal lines. In the intermediate layers (conv3_1, conv4_1), we start to see more complex patterns. Then concepts such as 'eyes' (filter 45) and 'entrances (?)' (filter 271) seem to appear in the last layer (conv5_1).

Refer to the notebooks below for more examples:

Talks & blog posts

Papers

Inspiration

Citation

Misa Ogura. (2019, July 8). MisaOgura/flashtorch: 0.0.8 (Version v0.0.8). Zenodo. http://doi.org/10.5281/zenodo.3271410

Say Thanks!

Author

Misa Ogura

R&D Software Engineer @ BBC

Co-founder of Women Driven Development

Github | Medium | twitter | LinkedIn

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for flashtorch, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size flashtorch-0.1.0.tar.gz (30.1 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page