Vision Unlearning

Documentation

Installation

pip install vision-unlearning

Compatible with python 3.10 to 3.12.

What is Vision Unlearning?

Vision Unlearning provides a standard interface for unlearning algorithms, datasets, metrics, and evaluation methodologies commonly used in Machine Unlearning for vision-related tasks, such as image classification and image generation.

It bridges the gap between research/theory and engineering/practice, making it easier to apply machine unlearning techniques effectively.

Vision Unlearning is designed to be:

• Easy to use
• Easy to extend
• Architecture-agnostic
• Application-agnostic

Who is it for?

Researchers

For Machine Unlearning researchers, Vision Unlearning helps with:

• Using the same data splits as other works, including the correct segmentation of forget-retain data and generating data with the same prompts.
• Choosing the appropriate metrics for each task.
• Configuring evaluation setups in a standardized manner.

Practitioners

For practitioners, Vision Unlearning provides:

• Easy access to state-of-the-art unlearning algorithms.
• A standardized interface to experiment with different algorithms.

Tutorials

• Replace George W. Bush by Tony Blair using FADE
• Replace George W. Bush by Tony Blair using FADE sparse-per-module
• Replace George W. Bush by Tony Blair using FADE sparse-per-weigth
• Forget cat using UCE (with hyperparam tunning)
• Forget church using Munba

The source code for these tutorials is in tutorials/, but their outputs were cleaned to avoid burdening the repo. The links above contain Google Drive stored executions with the full outputs.

For developers: every time there is a relevant modification in the codebase, please run the affected tutorials, save the notebook to Drive, clear the output before commiting.

Benchmarks

Vision-Unlearning provides easy access to evaluation benchmarks.

Across all benchmarks, a standardized set of files/media/metadata/content is provided following the structure described in the Appendix 2 of I-CARE paper. As such, they are compatible with Forgety.

I-CARE

Even though the feasibility demonstration of the I-CARE methodology is not per se a full-fledged benchmark, it is exposed by Vision-Unlearning as one. Three independent tasks are analyzed (forgetting people, scenes, and dog breeds) across three unlearning methods from the state-of-the-art (UCE, SPARE, MUNBa). Data, models, metrics and computed results are provided its HuggingFace repository. See code details in vision_unlearning/benchmarks/I_care.

This work analyzes how unlearning one entity (the emitter) affects the performance on other closely-related entities (the receivers). Each task is defined by carefully selecting a diverse and representative set of entities (concepts that will undergo unlearning). Each entity is annotated with relevant attributes, and separately unlearned using different unlearning methods. Each unlearned model is then used to generate images for all entities, allowing fine-grained analysis of the effects caused by the unlearning process. Last but not least, all entities contain the same amount of images and were carefully selected so as to be balanced across at least 2 attributes.

Unlearn Canvas

TODO... still under construction...

For more information on the benchmark itself, please refer to their repository https://github.com/OPTML-Group/UnlearnCanvas.

Data, models, metrics and computed results used by Vision-Unlearning are [or will be] provided this HuggingFace repository. See code details in vision_unlearning/benchmarks/u_care.

Holistic Unlearning Benchmark (HUB)

TODO... still under construction...

For more information on the benchmark itself, please refer to their paper: https://arxiv.org/abs/2410.05664

Data, models, metrics and computed results used by Vision-Unlearning are [or will be] provided this HuggingFace repository. See code details in vision_unlearning/benchmarks/hub_care.

Forgety

Forgety is a related project built on top of this library, providing a web-based graphical UI.

Main Interfaces

Vision Unlearning standardizes the following components:

• Metric: Evaluates a model (e.g., FID, CLIP Score, MIA, NudeNet, etc.).
• Unlearner: Encapsulates the unlearning algorithm.
• Dataset: Encapsulates the dataset, including data splitting.

Additionally, common tasks and evaluation setups are provided as example notebooks. Several platform integrations, such as Hugging Face and Weights & Biases, are also included.

Citation

We don't have yet a paper specifically about the library. Instead, please cite the Zenodo DOI that releases the data for the testbeds:

@dataset{vision_unlearning_evaluation_testbeds,
  author       = {Benitez Pereira, Leonardo Santiago and
                  Mola, Natnael and
                  R. Kelsch, Carolina and
                  Vaze, Soham},
  title        = {Vision Unlearning Evaluation Testbeds},
  month        = feb,
  year         = 2026,
  publisher    = {Zenodo},
  version      = {0.1.0},
  doi          = {10.5281/zenodo.18649818},
  url          = {https://doi.org/10.5281/zenodo.18649818},
}