persistable-clustering 0.5.1

Density-based clustering for exploratory data analysis based on multi-parameter persistence

Persistent and stable clustering (Persistable) is a density-based clustering algorithm intended for exploratory data analysis. What distinguishes Persistable from other clustering algorithms is its visualization capabilities. Persistable's interactive mode lets you visualize multi-scale and multi-density cluster structure present in the data. This is used to guide the choice of parameters that lead to the final clustering.

Usage

Here is a brief outline of the main functionality; see the documentation for details, including the API reference.

In order to run Persistable's interactive mode from a Jupyter notebook, run the following in a Jupyter cell:

import persistable
from sklearn.datasets import make_blobs

X = make_blobs(2000, centers=4, random_state=1)[0]

p = persistable.Persistable(X)
pi = persistable.PersistableInteractive(p)
pi.start_ui()

The last command returns the port in localhost serving the UI, which is 8050 by default. Now go to localhost:8050 in your web browser to access the graphical user interface:

After choosing your parameters using the user interface, you can get your clustering in another Jupyter cell by running:

clustering_labels = pi.cluster()

Note: You may use pi.start_ui(jupyter_mode="inline") to have the graphical user interface display directly in the Jupyter notebook!

Installing

Make sure you are using Python 3. Persistable depends on the following python packages, which will be installed automatically when you install with pip: numpy, scipy, scikit-learn, cython, plotly, dash, diskcache, multiprocess, psutil. To install from pypi, simply run the following:

pip install persistable-clustering

Documentation and support

You can find the documentation at persistable.readthedocs.io. If you have further questions, please open an issue and we will do our best to help you. Please include as much information as possible, including your system's information, warnings, logs, screenshots, and anything else you think may be of use. If you do not wish to open an issue, you are also welcome to contact Luis Scoccola directly. Please be patient if it takes us a bit to get back to you.

Running the tests

You can run the tests by running the following commands from the root directory of a clone of this repository. If a test fails, please report a bug, trying to include as much information as possible, including your system's information, warnings, logs, screenshots, and anything else you think may be of use.

pip install pytest playwright pytest-playwright
python -m playwright install --with-deps
pip install -r requirements.txt
python -m setup build_ext --inplace
pytest .

Details about theory and implementation

Persistable is based on multi-parameter persistence [4], a method from topological data analysis. The theory behind Persistable is developed in [1], while this implementation uses the high performance algorithms for density-based clustering developed in [2] and implemented in [3]. Persistable's interactive mode is inspired by RIVET [5] and is implemented in Dash.

Contributing

To contribute, you can fork the project, make your changes, and submit a pull request. You may want to contact Luis Scoccola first, to make sure your work does not overlap with ongoing work.

Authors

Luis Scoccola and Alexander Rolle.

Citing

If you use this package in your work, you may cite the corresponding paper using the following bibtex entry:

@article{Scoccola2023,
    doi = {10.21105/joss.05022},
    url = {https://doi.org/10.21105/joss.05022},
    year = {2023},
    publisher = {The Open Journal},
    volume = {8},
    number = {83},
    pages = {5022},
    author = {Luis Scoccola and Alexander Rolle},
    title = {Persistable: persistent and stable clustering},
    journal = {Journal of Open Source Software}
}

References

[1] Stable and consistent density-based clustering. A. Rolle and L. Scoccola. arXiv:2005.09048

[2] Accelerated Hierarchical Density Based Clustering. L. McInnes, J. Healy. 2017 IEEE International Conference on Data Mining Workshops (ICDMW), IEEE, pp 33-42. 2017

[3] hdbscan: Hierarchical density based clustering. L. McInnes, J. Healy, S. Astels. Journal of Open Source Software, The Open Journal, volume 2, number 11. 2017

[4] An Introduction to Multiparameter Persistence. M. B. Botnan, M. Lesnick. Proceedings of the 2020 International Conference on Representations of Algebras. 2022

[5] RIVET. The RIVET Developers. [Git] [docs]

License

This software is published under the 3-clause BSD license.