sortedness
sortedness is the level of agreement between two points regarding to how they rank all remaining points in a dataset.
This is valid even for points from different spaces, enabling the measurement of the quality of data transformation processes, often dimensionality reduction.
It is less sensitive to irrelevant distortions, and return values in a more meaningful interval, than Kruskal stress formula I.
This Python library / code provides a reference implementation for the functions presented here (paper unavailable until publication).
Overview
Local variants return a value for each provided point. The global variant returns a single value for all points. Any local variant can be used as a global measure by taking the mean value.
Local variants: sortedness(X, X_), pwsortedness(X, X_), rsortedness(X, X_).
Global variant: global_sortedness(X, X_).
Python installation
from package through pip
# Set up a virtualenv.
python3 -m venv venv
source venv/bin/activate
# Install from PyPI
pip install -U sortedness
from source
git clone https://github.com/sortedness/sortedness
cd sortedness
poetry install
Examples
Sortedness
import numpy as np
from numpy.random import permutation
from sklearn.decomposition import PCA
from sortedness import sortedness
# Some synthetic data.
mean = (1, 2)
cov = np.eye(2)
rng = np.random.default_rng(seed=0)
original = rng.multivariate_normal(mean, cov, size=12)
projected2 = PCA(n_components=2).fit_transform(original)
projected1 = PCA(n_components=1).fit_transform(original)
np.random.seed(0)
projectedrnd = permutation(original)
# Print `min`, `mean`, and `max` values.
s = sortedness(original, original)
print(min(s), sum(s) / len(s), max(s))
"""
1.0 1.0 1.0
"""
s = sortedness(original, projected2)
print(min(s), sum(s) / len(s), max(s))
"""
1.0 1.0 1.0
"""
s = sortedness(original, projected1)
print(min(s), sum(s) / len(s), max(s))
"""
0.3934632246658146 0.7565797804350681 0.944810120533741
"""
s = sortedness(original, projectedrnd)
print(min(s), sum(s) / len(s), max(s))
"""
-0.6483054795666044 -0.09539895194976367 0.3970195075915949
"""
# Single point fast calculation.
s = sortedness(original, projectedrnd, 2)
print(s)
"""
0.23107954749077175
"""
Pairwise sortedness
import numpy as np
from numpy.random import permutation
from sklearn.decomposition import PCA
from sortedness import pwsortedness
# Some synthetic data.
mean = (1, 2)
cov = np.eye(2)
rng = np.random.default_rng(seed=0)
original = rng.multivariate_normal(mean, cov, size=12)
projected2 = PCA(n_components=2).fit_transform(original)
projected1 = PCA(n_components=1).fit_transform(original)
np.random.seed(0)
projectedrnd = permutation(original)
# Print `min`, `mean`, and `max` values.
s = pwsortedness(original, original)
print(min(s), sum(s) / len(s), max(s))
"""
1.0 1.0 1.0
"""
s = pwsortedness(original, projected2)
print(min(s), sum(s) / len(s), max(s))
"""
1.0 1.0 1.0
"""
s = pwsortedness(original, projected1)
print(min(s), sum(s) / len(s), max(s))
"""
0.649315577592 0.7534291438324999 0.834601601062
"""
s = pwsortedness(original, projectedrnd)
print(min(s), sum(s) / len(s), max(s))
"""
-0.168611098044 -0.07988253899783333 0.14442446342
"""
# Single point fast calculation.
s = pwsortedness(original, projectedrnd, 2)
print(s)
"""
0.036119718802
"""
Global pairwise sortedness
import numpy as np
from numpy.random import permutation
from sklearn.decomposition import PCA
from sortedness import global_pwsortedness
# Some synthetic data.
mean = (1, 2)
cov = np.eye(2)
rng = np.random.default_rng(seed=0)
original = rng.multivariate_normal(mean, cov, size=12)
projected2 = PCA(n_components=2).fit_transform(original)
projected1 = PCA(n_components=1).fit_transform(original)
np.random.seed(0)
projectedrnd = permutation(original)
# Print measurement result and p-value.
s = global_pwsortedness(original, original)
print(list(s))
"""
[1.0, 3.6741408919675163e-93]
"""
s = global_pwsortedness(original, projected2)
print(list(s))
"""
[1.0, 3.6741408919675163e-93]
"""
s = global_pwsortedness(original, projected1)
print(list(s))
"""
[0.7715617715617715, 5.240847664048334e-20]
"""
s = global_pwsortedness(original, projectedrnd)
print(list(s))
"""
[-0.06107226107226107, 0.46847188611226276]
"""
** Copyright (c) 2023. Davi Pereira dos Santos and Tacito Neves**
TODO
Future work address handling large datasets: approximate sortedness value, and size-insensitive weighting scheme.
Reference
Please use the following reference to cite this work:
@inproceedings {10.2312:eurova.20231093,
booktitle = {EuroVis Workshop on Visual Analytics (EuroVA)},
editor = {Angelini, Marco and El-Assady, Mennatallah},
title = {{Nonparametric Dimensionality Reduction Quality Assessment based on Sortedness of Unrestricted Neighborhood}},
author = {Pereira-Santos, Davi and Neves, Tácito Trindade Araújo Tiburtino and Carvalho, André C. P. L. F. de and Paulovich, Fernando V.},
year = {2023},
publisher = {The Eurographics Association},
ISSN = {2664-4487},
ISBN = {978-3-03868-222-6},
DOI = {10.2312/eurova.20231093}
}
Grants
This work was supported by Wellcome Leap 1kD Program; São Paulo Research Foundation (FAPESP) - grant 2020/09835-1; Canadian Institute for Health Research (CIHR) Canadian Research Chairs (CRC) stipend [award number 1024586]; Canadian Foundation for Innovation (CFI) John R. Evans Leaders Fund (JELF) [grant number 38835]; Dalhousie Medical Research Fund (DMRF) COVID-19 Research Grant [grant number 603082]; and the Canadian Institute for Health Research (CIHR) Project Grant [award number 177968].
1 maintainer
Metadata
1 maintainer
Release files for sortedness 2.231107.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| sortedness-2.231107.1.tar.gz | 734.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| sortedness-2.231107.1-cp310-cp310-manylinux_2_35_x86_64.whl | CPython 3.10 | CPython 3.10 | Linux glibc 2.35+ x86-64 | Details |
Total release size: 1.5 MB
Release files / sortedness-2.231107.1.tar.gz
| Download URL | sortedness-2.231107.1.tar.gz |
|---|---|
| Size | 734.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6e42bc682d52610f18eed4c3288c2fc7c2d136d9a65e54253db777ad55df1119
|
|
BLAKE2b-256 checksum How to use checksums |
b2191898f8ba60b8ea5e5d404485f4eba4f26cdb4da467698754a69ba006f4f3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.5.1 CPython/3.10.6 Linux/6.2.0-35-generic
|
Release files / sortedness-2.231107.1-cp310-cp310-manylinux_2_35_x86_64.whl
| Download URL | sortedness-2.231107.1-cp310-cp310-manylinux_2_35_x86_64.whl |
|---|---|
| Size | 747.8 kB |
| Tags | CPython 3.10 Linux glibc 2.35+ x86-64 |
|
SHA-256 checksum How to use checksums |
fdb1bcc8afb99ed18712833b64aaf7224abd3a8a085e66704931d5f0edbfe7b3
|
|
BLAKE2b-256 checksum How to use checksums |
6f4974a43906a61d2199b72bb2a42df056c37bdb84f407abb37ba1f11a843def
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.5.1 CPython/3.10.6 Linux/6.2.0-35-generic
|
