Layer-Wise Relevance Propagation

PyTorch implementation of Layer-wise Relevance Propagation (LRP) algorithm together with quantitative evaluation metrics to compare heatmap explanations objectively.

Special thanks to Dr. Grégoire Montavon for his insights, which shaped the development of this project.

Features

Explainability:

• Layer-wise Relevance Propagation (LRP)

Quantitative Evaluation:

• Pixel Flipping (PF), also known as Region Perturbation (RP)
  • Perturbation modes:
    • inpainting
    • random
  • Sort objectives:
    • most relevant first (MoRF), also known as activation curve
    • least relevant first (LRF), also known as pruning curve
    • random
• AUC (Area Under the Curve), also known as AUAC (Area Under the Activation Curve) or AU-MSE (Area Under the Mean Squared Error Curve) depending on the sort objective

Showcase

LRP (lrp-tutorial composite) and PF with inpainting perturbation mode and sort objective MoRF.

Classification scores of castle image with inpainting perturbation mode

Number of simultaneous flips per perturbation step during Pixel-Flipping.

Requirements

• python3 >= 3.9

Installation

python3 -m pip install lrp-pf-auc

The PyPI distribution lrp-pf-auc provides the following two packages: lrp and pf.

The name lrp-pf-auc stands for Layer-wise Relevance Propagation (LRP), Pixel Flipping (PF), and Area Under the Curve (AUC) respectively.

Usage

Refer to demo.ipynb for an example of Layer-wise Relevance Propagation (LRP), Pixel-Flipping (PF) and Area under the Curve (AUC).

Feel free to check out the Jupyter notebooks under experiments/notebooks for a chronological overview of the project.

Related Projects

• Sequential LRP implementation: gmontavon/lrp-tutorial

  Tutorial on how to implement LRP

• Updated version of gmontavon/lrp-tutorial: rodrigobdz/lrp-tutorial
• Forward-hook LRP implementation: chr5tphr/zennit

  Implementation of LRP-based methods in PyTorch

• innvestigate-based LRP implementation: moboehle/Pytorch-LRP
• Caffe-based LRP implementation: lrp_toolbox
• Pixel-Flipping and Region Perturbation implementation: understandable-machine-intelligence-lab/Quantus

Citation

Cite as:

• Plaintext:

  Rodrigo Bermúdez Schettino. (2022). rodrigobdz/lrp: v0.1.6 (v0.1.6). Zenodo. https://doi.org/10.5281/zenodo.6821295

• BibTeX:

  @software{rodrigo_bermudez_schettino_2022_6821295,
    author       = {Rodrigo Bermúdez Schettino},
    title        = {rodrigobdz/lrp: v0.1.6},
    month        = jul,
    year         = 2022,
    publisher    = {Zenodo},
    version      = {v0.1.6},
    doi          = {10.5281/zenodo.6821295},
    url          = {https://doi.org/10.5281/zenodo.6821295}
  }
  

Credits

• The structure of this readme is based on minimal-readme

• Scripts follow Personal Shell Style Guide

• The lrp package uses two customized files originally from chr5tphr/zennit.

• The syntax for defining custom composites is inspired by this discussion on zennit's repo.

This implementation is based on insights from:

• LRP overview paper

  G. Montavon, A. Binder, S. Lapuschkin, W. Samek, K.-R. Müller Layer-wise Relevance Propagation: An Overview in Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer LNCS, vol. 11700, 2019

• Original LRP paper

  S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, W. Samek On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation PloS ONE 10 (7), e0130140, 2015

• ECML/PKDD 2020 Tutorial: Explainable AI for Deep Networks: Basics and Extensions (Part 3)

License

MIT © rodrigobdz