Clustering based on density with variable density clusters
HDBSCAN - Hierarchical Density-Based Spatial Clustering of Applications with Noise. Performs DBSCAN over varying epsilon values and integrates the result to find a clustering that gives the best stability over epsilon. This allows HDBSCAN to find clusters of varying densities (unlike DBSCAN), and be more robust to parameter selection.
- Based on the paper:
- R. Campello, D. Moulavi, and J. Sander, Density-Based Clustering Based on Hierarchical Density Estimates In: Advances in Knowledge Discovery and Data Mining, Springer, pp 160-172. 2013
Notebooks comparing HDBSCAN to other clustering algorithms, explaining how HDBSCAN works and comparing performance with other python clustering implementations are available.
How to use HDBSCAN
The hdbscan package inherits from sklearn classes, and thus drops in neatly next to other sklearn clusterers with an identical calling API. Similarly it supports input in a variety of formats: an array (or pandas dataframe, or sparse matrix) of shape (num_samples x num_features); an array (or sparse matrix) giving a distance matrix between samples.
import hdbscan clusterer = hdbscan.HDBSCAN(min_cluster_size=10) cluster_labels = clusterer.fit_predict(data)
Significant effort has been put into making the hdbscan implementation as fast as possible. It is more than twice as fast as the reference implementation in Java and is currently faster highly optimized single linkage implementations in C and C++. version 0.3 performance can be seen in this notebook . Version 0.4 brings a new minimum spanning tree algorithm and dramatic speedups, particularly for low dimensional data.
The hdbscan package comes equipped with visualization tools to help you understand your clustering results. After fitting data the clusterer object has attributes for:
- The condensed cluster hierarchy
- The robust single linkage cluster hierarchy
- The reachability distance minimal spanning tree
All of which come equipped with methods for plotting and converting to Pandas or NetworkX for further analysis. See the notebook on how HDBSCAN works for examples and further details.
The clusterer objects also have an attribute providing cluster membership strengths, resulting in optional soft clustering (and no further compute expense)
pip install hdbscan
For a manual install get this package:
wget https://github.com/lmcinnes/hdbscan/archive/master.zip unzip master.zip rm master.zip cd hdbscan-master
Install the requirements
sudo pip install -r requirements.txt
Install the package
python setup.py install
The hdbscan package is BSD licensed. Enjoy.
Release history Release notifications
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size hdbscan-0.4.1.tar.gz (32.8 kB)||File type Source||Python version None||Upload date||Hashes View hashes|