Skip to main content

Automatic Concept Extraction with CRAFT

Project description

👋 CRAFT: Concept Recursive Activation FacTorization for Explainability (CVPR 2023)

This repository contains code for the paper:

CRAFT: Concept Recursive Activation FacTorization for Explainability, Thomas Fel*, Agustin Picard*, Louis Bethune*, Thibaut Boissin*, David Vigouroux, Julien Colin, Rémi Cadène, Thomas Serre. CVPR 2023, [arXiv].

The code is implemented and available for Pytorch & Tensorflow. A notebook for each of them is available:

🚀 Quick Start

Craft requires a version of python higher than 3.6 and several libraries like Numpy, also you will need either Tensorflow or Torch. Installation can be done using Pypi:

pip install Craft-xai

Now that Craft is installed, here is the basic example of what you can do. The API, whether for Tensorflow or Pytorch, is similar and only requires two hyperparameters. First, you need to load your models and a set of images from a class you want to explain (generally, try to have at least 500 images).

Once you have that, split your model into two parts (see the notebooks if necessary) to have two functions: $g$, which maps from the input to the feature space, and $h$, which maps from the feature space to your logits. Once you have done this, you are ready to instantiate CRAFT.

from craft.craft_torch import Craft
# or
#from craft.craft_tf import Craft


craft = Craft(input_to_latent=g,
              latent_to_logit=h,
              number_of_concepts=10,
              patch_size=64,
              batch_size=64)

Now, you can fit CRAFT with your preprocessed images (make sure they are preprocessed according to your model).

crops, crops_u, w = craft.fit(images_preprocessed)
importances = craft.estimate_importance(images_preprocessed, class_id=class_id) # the logit you want to explain

That's it! To learn how to visualize the results, refer to the notebooks that explain how to make the most of all the information returned by CRAFT.

Citation

@inproceedings{fel2023craft,
      title={CRAFT: Concept Recursive Activation FacTorization for Explainability},
      author={Thomas, Fel and Agustin, Picard and Louis, Bethune and Thibaut, Boissin and David, Vigouroux and Julien, Colin and Rémi, Cadène and Thomas, Serre},
      year={2023},
      booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}
}

More about CRAFT

The code for the metrics and the other attribution methods used in the paper come from the Xplique toolbox.

Additionally, we have created a website called the LENS Project, which features the 1000 classes of ImageNet.

Authors of the code

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Craft-xai-0.0.3.tar.gz (14.1 kB view details)

Uploaded Source

Built Distribution

Craft_xai-0.0.3-py3-none-any.whl (18.6 kB view details)

Uploaded Python 3

File details

Details for the file Craft-xai-0.0.3.tar.gz.

File metadata

  • Download URL: Craft-xai-0.0.3.tar.gz
  • Upload date:
  • Size: 14.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.4

File hashes

Hashes for Craft-xai-0.0.3.tar.gz
Algorithm Hash digest
SHA256 d819ab550b471a8f3de9898be88660edf0ec62c8ce71f47481be1a7ef45edbd0
MD5 7a45e21f1e878431f9508138be46c099
BLAKE2b-256 b61189141a295d890df2db91e9ec8734664033cdc3b44400983190d1e1704cd6

See more details on using hashes here.

File details

Details for the file Craft_xai-0.0.3-py3-none-any.whl.

File metadata

  • Download URL: Craft_xai-0.0.3-py3-none-any.whl
  • Upload date:
  • Size: 18.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.4

File hashes

Hashes for Craft_xai-0.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 20dc71be83e80e4f68a05ddb46bec4ed652230f980e6ccb86df72922bd3b0f21
MD5 717089aee4129bf796848bdd58f0e2fb
BLAKE2b-256 b9dc632203fad375529f990290778f861b2623f45bbfc35397edeefc5cb32f46

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page