cognitivefactory-interactive-clustering-gui 0.2.1

An annotation tool for NLP data based on Interactive Clustering methodology.

Interactive Clustering GUI

An annotation tool for NLP data based on Interactive Clustering methodology.

Quick description

Interactive clustering is a method intended to assist in the design of a training data set.

This iterative process begins with an unlabeled dataset, and it uses a sequence of two substeps :

1. the user defines constraints on data sampled by the computer ;

2. the computer performs data partitioning using a constrained clustering algorithm.

Thus, at each step of the process :

• the user corrects the clustering of the previous steps using constraints, and

• the computer offers a corrected and more relevant data partitioning for the next step.

This web application implements this annotation methodology with several features:

• data preprocessing and vectorization in order to reduce noise in data;

• constrainted clustering in order to automatically partition the data;

• constraints sampling in order to select the most relevant data to annotate;

• binary constraints annotation in order to correct clustering relevance;

• annotation review and conflicts analysis in order to improve constraints consistency.

• For more details, read the Documentation and the articles in the References section.

Documentation

• Main documentation

Requirements

Interactive Clustering GUI requires Python 3.7 or above.

To install Python 3.7, I recommend using pyenv.

# install pyenv
git clone https://github.com/pyenv/pyenv ~/.pyenv

# setup pyenv (you should also put these three lines in .bashrc or similar)
export PATH="${HOME}/.pyenv/bin:${PATH}"
export PYENV_ROOT="${HOME}/.pyenv"
eval "$(pyenv init -)"

# install Python 3.7
pyenv install 3.7

# make it available globally
pyenv global system 3.7

Installation

With pip:

# install package
python3 -m pip install cognitivefactory-interactive-clustering-gui

# install spacy language model dependencies (the one you want, with version "3.1.x")
python3 -m spacy download fr_core_news_md-3.1.0 --direct

With pipx:

# install pipx
python3 -m pip install --user pipx

# install package
pipx install --python python3 cognitivefactory-interactive-clustering-gui

# install spacy language model dependencies (the one you want, with version "3.1.x")
python3 -m spacy download fr_core_news_md-3.1.0 --direct

Development

To work on this project or contribute to it, please read the Copier PDM documentation.

Quick setup and help

Get the code and prepare the environment:

git clone https://github.com/cognitivefactory/interactive-clustering-gui/
cd interactive-clustering-gui
make setup

Show the help:

make help  # or just make

For more details, read the Contributing documentation.

References

• Interactive Clustering:

  • First presentation: Schild, E., Durantin, G., Lamirel, J.C., & Miconi, F. (2021). Conception itérative et semi-supervisée d'assistants conversationnels par regroupement interactif des questions. In EGC 2021 - 21èmes Journées Francophones Extraction et Gestion des Connaissances. Edition RNTI. ⟨hal-03133007⟩.
  • Theoretical study: Schild, E., Durantin, G., Lamirel, J., & Miconi, F. (2022). Iterative and Semi-Supervised Design of Chatbots Using Interactive Clustering. International Journal of Data Warehousing and Mining (IJDWM), 18(2), 1-19. http://doi.org/10.4018/IJDWM.298007. ⟨hal-03648041⟩.
  • Methodological discussion: Schild, E., Durantin, G., & Lamirel, J.C. (2021). Concevoir un assistant conversationnel de manière itérative et semi-supervisée avec le clustering interactif. In Atelier - Fouille de Textes - Text Mine 2021 - En conjonction avec EGC 2021. ⟨hal-03133060⟩.
  • Implementation: Schild, E. (2021). cognitivefactory/interactive-clustering. Zenodo. https://doi.org/10.5281/zenodo.4775251.

• Web application:

  • FastAPI: https://fastapi.tiangolo.com/

How to cite

Schild, E. (2021). cognitivefactory/interactive-clustering-gui. Zenodo. https://doi.org/10.5281/zenodo.4775270.