divik
Python implementation of Divisive iK-means (DiviK) algorithm.
Tools within this package
- Clustering at your command line with fit-clusters
- Set of algorithm implementations for unsupervised analyses
- Clustering
- DiviK - hands-free clustering method with built-in feature selection
- K-Means with Dunn method for selecting the number of clusters
- K-Means with GAP index for selecting the number of clusters
- Modular K-Means implementation with custom distance metrics and initializations
- Feature extraction
- PCA with knee-based components selection
- Locally Adjusted RBF Spectral Embedding
- Feature selection
- EXIMS
- Gaussian Mixture Model based data-driven feature selection
- High Abundance And Variance Selector - allows you to select highly variant features above noise level, based on GMM-decomposition
- Outlier based Selector
- Outlier Abundance And Variance Selector - allows you to select highly variant features above noise level, based on outlier detection
- Percentage based Selector - allows you to select highly variant features above noise level with your predefined thresholds for each
- Sampling
- StratifiedSampler - generates samples of fixed number of rows from given dataset
- UniformPCASampler - generates samples of random observations within boundaries of an original dataset, and preserving the rotation of the data
- UniformSampler - generates samples of random observations within boundaries of an original dataset
- Clustering
Installation
Docker
The recommended way to use this software is through
Docker. This is the most convenient way, if you want
to use divik application.
To install latest stable version use:
docker pull gmrukwa/divik
Python package
Prerequisites for installation of base package:
- Python 3.7 / 3.8 / 3.9
- compiler capable of compiling the native C code and OpenMP support
Installation of OpenMP for Ubuntu / Debian
You should have it already installed with GCC compiler, but if somehow not, try the following:
sudo apt-get install libgomp1
Installation of OpenMP for Mac
OpenMP is available as part of LLVM. You may need to install it with conda:
conda install -c conda-forge "compilers>=1.0.4,!=1.1.0" llvm-openmp
Installation of dependencied on Mac
You may see messages that some dependencies are invalid for the platform. It is a known bug, with a workaround.
Use:
SYSTEM_VERSION_COMPAT=0 pip install divik
DiviK Installation
Having prerequisites installed, one can install latest base version of the package:
pip install divik
If you want to have compatibility with
gin-config, you can install
necessary extras with:
pip install divik[gin]
Note: Remember about \ before [ and ] in zsh shell.
You can install all extras with:
pip install divik[all]
High-Volume Data Considerations
If you are using DiviK to run the analysis that could fail to fit RAM of your computer, consider disabling the default parallelism and switch to dask. It's easy to achieve through configuration:
- set all parameters named
n_jobsto1; - set all parameters named
allow_dasktoTrue.
Note: Never set n_jobs>1 and allow_dask=True at the same time, the
computations will freeze due to how multiprocessing and dask handle
parallelism.
Known Issues
Segmentation Fault
It can happen if the he gamred_native package (part of divik package) was
compiled with different numpy ABI than scikit-learn. This could happen if you
used different set of compilers than the developers of the scikit-learn
package.
In such a case, a handler is defined to display the stack trace. If the trace
comes from _matlab_legacy.py, the most probably this is the issue.
To resolve the issue, consider following the installation instructions once again. The exact versions get updated to avoid the issue.
Contributing
Contribution guide will be developed soon.
Format the code with:
isort -m 3 --fgw 3 --tc .
black -t py36 .
References
This software is part of contribution made by Data Mining Group of Silesian University of Technology, rest of which is published here.
1 maintainer
Metadata
1 maintainer
Release files for divik 3.2.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| divik-3.2.4.tar.gz | 83.2 kB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| divik-3.2.4-cp39-cp39-win_amd64.whl | CPython 3.9 | CPython 3.9 | Windows x86-64 | Details |
| divik-3.2.4-cp39-cp39-manylinux_2_35_x86_64.whl | CPython 3.9 | CPython 3.9 | Linux glibc 2.35+ x86-64 | Details |
| divik-3.2.4-cp38-cp38-win_amd64.whl | CPython 3.8 | CPython 3.8 | Windows x86-64 | Details |
| divik-3.2.4-cp38-cp38-manylinux_2_35_x86_64.whl | CPython 3.8 | CPython 3.8 | Linux glibc 2.35+ x86-64 | Details |
| divik-3.2.4-cp37-cp37m-win_amd64.whl | CPython 3.7 | CPython 3.7 pymalloc | Windows x86-64 | Details |
| divik-3.2.4-cp37-cp37m-manylinux_2_35_x86_64.whl | CPython 3.7 | CPython 3.7 pymalloc | Linux glibc 2.35+ x86-64 | Details |
Total release size: 921.8 kB
Release files / divik-3.2.4.tar.gz
| Download URL | divik-3.2.4.tar.gz |
|---|---|
| Size | 83.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
62009c32c63c3cc15563c52c0cb6744e38a294fea8c529697cb0ff7ea5db44f3
|
|
BLAKE2b-256 checksum How to use checksums |
6e451a4e348b437fd15d46fd0e5eda2e879fd2551240e1f2931f582731b55639
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.11.9
|
Release files / divik-3.2.4-cp39-cp39-win_amd64.whl
| Download URL | divik-3.2.4-cp39-cp39-win_amd64.whl |
|---|---|
| Size | 113.9 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
2d5134940a61b2723e199ea5e8c15b71fc3cb4397b92f94bbd94c38979f850d3
|
|
BLAKE2b-256 checksum How to use checksums |
58789c4afabccb1c636604fb6ee05a022e2a706aa9c470fb6a9a3ae8de3ad5bf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.3 CPython/3.9.13 Windows/10
|
Release files / divik-3.2.4-cp39-cp39-manylinux_2_35_x86_64.whl
| Download URL | divik-3.2.4-cp39-cp39-manylinux_2_35_x86_64.whl |
|---|---|
| Size | 165.6 kB |
| Tags | CPython 3.9 Linux glibc 2.35+ x86-64 |
|
SHA-256 checksum How to use checksums |
a70483e2298f07281ae6e466e489dfa1d23c511df6bbeba4228b7a6de25ffd4d
|
|
BLAKE2b-256 checksum How to use checksums |
0dff7145acf26536da0e1c52d27b2becf325ba84196fc95645feb87a4819be92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.3 CPython/3.9.19 Linux/6.5.0-1022-azure
|
Release files / divik-3.2.4-cp38-cp38-win_amd64.whl
| Download URL | divik-3.2.4-cp38-cp38-win_amd64.whl |
|---|---|
| Size | 113.9 kB |
| Tags | CPython 3.8 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
13927ce0533479dec96170e30c338ab8fac58b8d979593065dd1be3736570b9a
|
|
BLAKE2b-256 checksum How to use checksums |
b8e285d08e25bebb34e6c3445c9869e474665121e6d8f393d64b2add96cc741e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.3 CPython/3.8.10 Windows/10
|
Release files / divik-3.2.4-cp38-cp38-manylinux_2_35_x86_64.whl
| Download URL | divik-3.2.4-cp38-cp38-manylinux_2_35_x86_64.whl |
|---|---|
| Size | 165.9 kB |
| Tags | CPython 3.8 Linux glibc 2.35+ x86-64 |
|
SHA-256 checksum How to use checksums |
972ab03cadcccf672c90b3fe86a9a12faf5f213d1435558df6eddc63877e3a65
|
|
BLAKE2b-256 checksum How to use checksums |
f4e7384147cba13d28318e970fe2384d50b431a2eb55516cef858a8da680da27
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.3 CPython/3.8.18 Linux/6.5.0-1022-azure
|
Release files / divik-3.2.4-cp37-cp37m-win_amd64.whl
| Download URL | divik-3.2.4-cp37-cp37m-win_amd64.whl |
|---|---|
| Size | 113.9 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
21c563aa5d37f685ab1282177087fd282bba4fe41a5f7b588c3619ad78dbc8f3
|
|
BLAKE2b-256 checksum How to use checksums |
84e5b82d9df92f9c1500f280c08f523f504d94d6c02bb8c1f2e84299f5318a56
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.5.1 CPython/3.7.9 Windows/10
|
Release files / divik-3.2.4-cp37-cp37m-manylinux_2_35_x86_64.whl
| Download URL | divik-3.2.4-cp37-cp37m-manylinux_2_35_x86_64.whl |
|---|---|
| Size | 165.5 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Linux glibc 2.35+ x86-64 |
|
SHA-256 checksum How to use checksums |
1a1f50ab2a5e03a5a99f9e5d69359b5cfa797d767d44f42ca0b82ac8b8e661aa
|
|
BLAKE2b-256 checksum How to use checksums |
376422b7bc4bd3eaf3c78419951de1925b10a3b9a35821eb150d37fcaf5b6399
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.5.1 CPython/3.7.17 Linux/6.5.0-1022-azure
|
