Neural network visualization toolkit for tf.keras
v0.7.0! In this release, the gradient calculation of ActivationMaximization is changed for the sake of fixing a critical problem. Although the calculation result are now a bit different compared to the past versions, you could avoid it by using legacy implementation as follows:
# from tf_keras_vis.activation_maximization import ActivationMaximization from tf_keras_vis.activation_maximization.legacy import ActivationMaximization
In addition to above, we've also fixed some problems related Regularizers. Although we newly provide
tf_keras_vis.activation_maximization.regularizers module that includes the regularizers whose bugs are fixed, like ActivationMaximization, you could also use legacy implementation as follows:
# from tf_keras_vis.activation_maximization.regularizers import Norm, TotalVariation2D from tf_keras_vis.utils.regularizers import Norm, TotalVariation2D
tf-keras-vis is a visualization toolkit for debugging
tf.keras.Model in Tensorflow2.0+.
Currently supported methods for visualization include:
- Feature Visualization
- Class Activation Maps
- Saliency Maps
tf-keras-vis is designed to be light-weight, flexible and ease of use. All visualizations have the features as follows:
- Support N-dim image inputs, that's, not only support pictures but also such as 3D images.
- Support batch wise processing, so, be able to efficiently process multiple input images.
- Support the model that have either multiple inputs or multiple outputs, or both.
- Support the mixed-precision model.
And in ActivationMaximization,
- Support Optimizers that are built to tf.keras.
Class Activation Map
The images above are generated by
The images above are generated by
ActivationMaximization (Visualizing Convolutional Filter)
import tensorflow as tf from tensorflow.keras.applications import VGG16 from matplotlib import pyplot as plt from tf_keras_vis.activation_maximization import ActivationMaximization from tf_keras_vis.activation_maximization.callbacks import Progress from tf_keras_vis.activation_maximization.input_modifiers import Jitter, Rotate2D from tf_keras_vis.activation_maximization.regularizers import TotalVariation2D, Norm from tf_keras_vis.utils.model_modifiers import ExtractIntermediateLayer, ReplaceToLinear from tf_keras_vis.utils.scores import CategoricalScore # Create the visualization instance. # All visualization classes accept a model and model-modifier, which, for example, # replaces the activation of last layer to linear function so on, in constructor. activation_maximization = \ ActivationMaximization(VGG16(), model_modifier=[ExtractIntermediateLayer('block5_conv3'), ReplaceToLinear()], clone=False) # You can use Score class to specify visualizing target you want. # And add regularizers or input-modifiers as needed. activations = \ activation_maximization(CategoricalScore(FILTER_INDEX), steps=200, input_modifiers=[Jitter(jitter=16), Rotate2D(degree=1)], regularizers=[TotalVariation2D(weight=1.0), Norm(weight=0.3, p=1)], optimizer=tf.keras.optimizers.RMSprop(1.0, 0.999), callbacks=[Progress()]) ## Since v0.6.0, calling `astype()` is NOT necessary. # activations = activations.astype(np.uint8) # Render plt.imshow(activations)
import numpy as np from matplotlib import pyplot as plt from matplotlib import cm from tf_keras_vis.gradcam_plus_plus import GradcamPlusPlus from tf_keras_vis.utils.model_modifiers import ReplaceToLinear from tf_keras_vis.utils.scores import CategoricalScore # Create GradCAM++ object gradcam = GradcamPlusPlus(YOUR_MODEL_INSTANCE, model_modifier=ReplaceToLinear(), clone=True) # Generate cam with GradCAM++ cam = gradcam(CategoricalScore(CATEGORICAL_INDEX), SEED_INPUT) ## Since v0.6.0, calling `normalize()` is NOT necessary. # cam = normalize(cam) plt.imshow(SEED_INPUT_IMAGE) heatmap = np.uint8(cm.jet(cam)[..., :3] * 255) plt.imshow(heatmap, cmap='jet', alpha=0.5) # overlay
Please see the guides below for more details:
Getting Started Guides
[NOTES] If you have ever used keras-vis, you may feel that tf-keras-vis is similar with keras-vis. Actually tf-keras-vis derived from keras-vis, and both provided visualization methods are almost the same. But please notice that tf-keras-vis APIs does NOT have compatibility with keras-vis.
- Python 3.7-3.10
$ pip install tf-keras-vis tensorflow
- Source (for development)
$ git clone https://github.com/keisen/tf-keras-vis.git $ cd tf-keras-vis $ pip install -e .[develop] tensorflow
- A Deep Learning Computer Vision library for easy data loading, model building and model interpretation with GradCAM/GradCAM++.
- With InceptionV3, ActivationMaximization doesn't work well, that's, it might generate meaninglessly blur image.
- With cascading model, Gradcam and Gradcam++ don't work well, that's, it might occur some error. So we recommend to use FasterScoreCAM in this case.
channels-firstmodels and data is unsupported.
- Visualizing multiple attention or activation images at once utilizing batch-system of model
- Define various score functions
- Visualizing attentions with multiple inputs models
- Visualizing attentions with multiple outputs models
- Advanced score functions
- Tuning Activation Maximization
- Visualizing attentions for N-dim image inputs
- We're going to add some methods such as below
- Deep Dream
- Style transfer
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for tf_keras_vis-0.8.1-py3-none-any.whl