🧼🔎 SelfClean

SelfClean Teaser

A holistic self-supervised data cleaning strategy to detect off-topic samples, near duplicates, and label errors.

Publications: SelfClean Paper (NeurIPS24) | Data Cleaning Protocol Paper (ML4H23@NeurIPS)

NOTE: Make sure to have git-lfs installed before pulling the repository to ensure the pre-trained models are pulled correctly (git-lfs install instructions).

This project is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International license.

Installation

Install SelfClean via PyPI:

# upgrade pip to its latest version
pip install -U pip

# install selfclean
pip install selfclean

# Alternatively, use explicit python version (XX)
python3.XX -m pip install selfclean

Getting Started

You can run SelfClean in a few lines of code:

from selfclean import SelfClean

selfclean = SelfClean(
    # displays the top-7 images from each error type
    # per default this option is disabled
    plot_top_N=7,
)

# run on pytorch dataset
issues = selfclean.run_on_dataset(
    dataset=copy.copy(dataset),
)
# run on image folder
issues = selfclean.run_on_image_folder(
    input_path="path/to/images",
)

# get the data quality issue rankings
df_near_duplicates = issues.get_issues("near_duplicates", return_as_df=True)
df_off_topic_samples = issues.get_issues("off_topic_samples", return_as_df=True)
df_label_errors = issues.get_issues("label_errors", return_as_df=True)

Examples: In examples/, we've provided some example notebooks where you will learn how to analyze and clean datasets using SelfClean. These examples analyze different benchmark datasets such as:

Imagenette 🖼️ (Open in NBViewer | GitHub | Colab)
Oxford-IIIT Pet 🐶 (Open in NBViewer | GitHub | Colab)

Also, check out our Kaggle notebook to see an illustration of how to get a gold medal for cleaning a competition dataset.

More Ressources

Reference

If you find this repository useful for your research, please cite the following work.

@article{groger_selfclean_2024,
  title        = {{Intrinsic Self-Supervision for Data Quality Audits}},
  shorttitle   = {{SelfClean}},
  author       = {Gr\"oger, Fabian and Lionetti, Simone and Gottfrois, Philippe and Gonzalez-Jimenez, Alvaro and Amruthalingam, Ludovic and Consortium, Labelling and Groh, Matthew and Navarini, Alexander A. and Pouly, Marc},
  year         = 2024,
  month        = 12,
  journal      = {Advances in Neural Information Processing Systems (NeurIPS)},
}

Development Environment

Run make for a list of possible targets.

Run these commands to install the requirements for the development environment:

make init
make install

To run linters on all files:

pre-commit run --all-files

We use the following packages for code and test conventions:

black for code style
isort for import sorting
pytest for running tests

Metadata

Release files for selfclean 0.0.39

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for selfclean 0.0.39
File	Size	Uploaded
selfclean-0.0.39.tar.gz	112.4 kB	Apr 28, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for selfclean 0.0.39
File	Interpreter	ABI	Platform	Reset
selfclean-0.0.39-py3-none-any.whl 169.8 kB Apr 28, 2026	Python 3	none	any	Details

Total release size: 282.2 kB

Release files / selfclean-0.0.39.tar.gz

Download URL	selfclean-0.0.39.tar.gz
Size	112.4 kB
Tags	Source
SHA-256 checksum How to use checksums	`273a53a8886b92b8e77d18461fd18492ffd368bc75d92773a85680b30432369f`
BLAKE2b-256 checksum How to use checksums	`3184f4e6bb241121f7253249390cd2f194e125d56006f0aed982ab8bd7667712`
Upload date	Apr 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.9.25`

Release files / selfclean-0.0.39-py3-none-any.whl

Download URL	selfclean-0.0.39-py3-none-any.whl
Size	169.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`71695670fca35fd5f6bc69d12c5a635ba16588dd5fc59b2d1ba59c6f2fb7e8d2`
BLAKE2b-256 checksum How to use checksums	`6abdb0461c4d684eca1e1a8a304a6bbbc14abd37a88d79d388e8215f4c02baed`
Upload date	Apr 28, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.9.25`