cognitivefactory-interactive-clustering-gui 0.4.0

A web application designed for NLP data annotation using Interactive Clustering methodology.

Interactive Clustering GUI

A web application designed for NLP data annotation using Interactive Clustering methodology.

Quick description

Interactive clustering is a method intended to assist in the design of a training data set.

This iterative process begins with an unlabeled dataset, and it uses a sequence of two substeps :

1. the user defines constraints on data sampled by the computer ;
2. the computer performs data partitioning using a constrained clustering algorithm.

Thus, at each step of the process :

• the user corrects the clustering of the previous steps using constraints, and
• the computer offers a corrected and more relevant data partitioning for the next step.

This web application implements this annotation methodology with several features:

• data preprocessing and vectorization in order to reduce noise in data;
• constrainted clustering in order to automatically partition the data;
• constraints sampling in order to select the most relevant data to annotate;
• binary constraints annotation in order to correct clustering relevance;
• annotation review and conflicts analysis in order to improve constraints consistency.

For more details, read the Documentation and the articles in the References section.

Documentation

• Main documentation

Requirements

Interactive Clustering GUI requires Python 3.7 or above.

To install Python 3.7, I recommend using pyenv.

# install pyenv
git clone https://github.com/pyenv/pyenv ~/.pyenv

# setup pyenv (you should also put these three lines in .bashrc or similar)
export PATH="${HOME}/.pyenv/bin:${PATH}"
export PYENV_ROOT="${HOME}/.pyenv"
eval "$(pyenv init -)"

# install Python 3.7
pyenv install 3.7

# make it available globally
pyenv global system 3.7

Installation

With pip:

# install package
python3 -m pip install cognitivefactory-interactive-clustering-gui

# install spacy language model dependencies (the one you want, with version "3.1.x")
python3 -m spacy download fr_core_news_md-3.1.0 --direct

With pipx:

# install pipx
python3 -m pip install --user pipx

# install package
pipx install --python python3 cognitivefactory-interactive-clustering-gui

# install spacy language model dependencies (the one you want, with version "3.1.x")
python3 -m spacy download fr_core_news_md-3.1.0 --direct

Run

To display the help message:

cognitivefactory-interactive-clustering-gui --help

To launch the web application:

cognitivefactory-interactive-clustering-gui  # launch on 127.0.0.1:8080

Then, go to one of the following pages in your browser:

• Welcome page (web application home): http://localhost:8080
• Swagger (interactive documentation): http://localhost:8080/docs

Development

To work on this project or contribute to it, please read the Copier PDM documentation.

Quick setup and help

Get the code and prepare the environment:

git clone https://github.com/cognitivefactory/interactive-clustering-gui/
cd interactive-clustering-gui
make setup

Show the help:

make help  # or just make

Launch the web application in debug mode:

make run  # launch on 127.0.0.1:8080

Then, go to one of the following pages in your browser:

• Welcome page (web application home): http://localhost:8080
• Swagger (interactive documentation): http://localhost:8080/docs

For more details, read the Contributing documentation.

References

• Interactive Clustering:

  • First presentation: Schild, E., Durantin, G., Lamirel, J.C., & Miconi, F. (2021). Conception itérative et semi-supervisée d'assistants conversationnels par regroupement interactif des questions. In EGC 2021 - 21èmes Journées Francophones Extraction et Gestion des Connaissances. Edition RNTI. ⟨hal-03133007⟩.
  • Theoretical study: Schild, E., Durantin, G., Lamirel, J., & Miconi, F. (2022). Iterative and Semi-Supervised Design of Chatbots Using Interactive Clustering. International Journal of Data Warehousing and Mining (IJDWM), 18(2), 1-19. http://doi.org/10.4018/IJDWM.298007. ⟨hal-03648041⟩.
  • Methodological discussion: Schild, E., Durantin, G., & Lamirel, J.C. (2021). Concevoir un assistant conversationnel de manière itérative et semi-supervisée avec le clustering interactif. In Atelier - Fouille de Textes - Text Mine 2021 - En conjonction avec EGC 2021. ⟨hal-03133060⟩.
  • Implementation: Schild, E. (2021). cognitivefactory/interactive-clustering. Zenodo. https://doi.org/10.5281/zenodo.4775251.

• Web application:

  • FastAPI: https://fastapi.tiangolo.com/

How to cite

Schild, E. (2021). cognitivefactory/interactive-clustering-gui. Zenodo. https://doi.org/10.5281/zenodo.4775270.