Skip to main content

Benchmark interpretability methods.

Project description

BIM - Benchmark Interpretability Method

This repository contains dataset, models, and metrics for benchmarking interpretability methods (BIM) described in paper:

  • Title: "BIM: Towards Quantitative Evaluation of Interpretability Methods with Ground Truth"
  • Authors: Sherry (Mengjiao) Yang, Been Kim

Upon using this library, please cite:

  title = {{BIM: Towards Quantitative Evaluation of Interpretability Methods with Ground Truth}},
  author = {Yang, Mengjiao and Kim, Been},
  year = {2019}


Run the following from the home directory of this repository to install python dependencies, download BIM models, download MSCOCO and MiniPlaces, and construct BIM dataset.

pip install bim
source scripts/
source scripts/
python scripts/


Images in data/obj and data/scene are the same but have object and scene labels respectively, as shown in the figure above. val_loc.txt records the top-left and bottom-right corner of the object and val_mask has the binary masks of the object in the validation set. Additional sets and their usage are described in the table below.

Name Training Validation Usage Description
obj 90,000 10,000 Model contrast Objects and scenes with object labels
scene 90,000 10,000 Model contrast & Input dependence Objects and scenes with scene labels
scene_only 90,000 10,000 Input dependence Scene-only images with scene labels
dog_bedroom - 200 Relative model contrast Dog in bedroom labeled as bedroom
bamboo_forest - 100 Input independence Scene-only images of bamboo forest
bamboo_forest_patch - 100 Input independence Bamboo forest with functionally insignificant dog patch


Models in models/obj, models/scene, and models/scene_only are trained on data/obj, data/scene, and data/scene_only respectively. Models in models/scenei for i in {1...10} are trained on images where dog is added to i scene classes, and the rest scene classes do not contain any added objects. All models are in TensorFlow's SavedModel format.


BIM metrics compare how interpretability methods perform across models (model contrast), across inputs to the same model (input dependence), and across functionally equivalent inputs (input independence).

Model contrast scores

Given images that contain both objects and scenes, model contrast measures the difference in attributions between the model trained on object labels and the model trained on scene labels.

Input dependence rate

Given a model trained on scene labels, input dependence measures the percentage of inputs where the addition of objects results in the region being attributed as less important.

Input independence rate

Given a model trained on scene-only images, input independence measures the percentage of inputs where a functionally insignificant patch (e.g., a dog) does not affect explanations significantly.

Evaluate saliency methods

To compute model contrast score (MCS) over randomly selected 10 images, you can run

python bim/ --metrics=MCS --num_imgs=10

To compute input dependence rate (IDR), change --metrics to IDR. To compute input independence rate (IIR), you need to first constructs a set of functionally insignificant patches by running

python scripts/

and then evaluate IIR by running

python bim/ --metrics=IIR --num_imgs=10

Evaluate TCAV

TCAV is a global concept attribution method whose MCS can be measured by comparing the TCAV scores of a particular object concept for the object model and the scene model. Run the following to compute the TCAV scores of the dog concept for the object model.

python bim/ --model=obj


This is not an officially supported Google product.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bim-0.4.tar.gz (11.1 kB view hashes)

Uploaded source

Built Distribution

bim-0.4-py3-none-any.whl (17.0 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page