Skip to main content

labelsim

Similarity measures between labels of multi-label data, computed on the contingency table (a, b, c, d) of each pair of labels.

The library gathers the 107 measures of the MultiLabelSimilaritiesMeasures repository (Gatto, 2021) and the normalized mutual information (Strehl and Ghosh, 2002), in two variants.

  • literature (default) applies the formula as published. The 107 measures yield 64 distinct formulas, because several names share the same formula, and the NMI completes the 65.
  • gatto2021 reproduces the original R code line by line, including where it diverges from the published formula. It is meant for comparison with results obtained with that code.

It is part of the undergraduate thesis Análise Comparativa de Medidas de Similaridade para Modelagem de Correlações em Classificação Multirrótulo (UFLA). The experiments of the thesis are in the Similarity-Measures repository, in R.

Installation

pip install -e .            # numpy and scipy
pip install -e ".[dev]"     # plus pandas, scikit-learn and pytest

Requires Python 3.10 or newer.

Usage

import numpy as np
import labelsim

# one row per instance, one column per label
Y = np.array([[1, 0, 1],
              [1, 1, 0],
              [0, 1, 1],
              [1, 0, 0]])

m = labelsim.similarity(Y, ["jaccard", "phi", "nmi"])
m["jaccard"]          # 3 x 3 matrix

# long format, one pair i < j per row (requires pandas)
labelsim.similarity(Y, ["jaccard", "phi"], output="long", labels=["x", "y", "z"])

# counts already computed
labelsim.from_contingency(a=10, b=5, c=3, d=982, measures=["yule_q", "sokal_michener"])

# a new formula, without touching the catalog
labelsim.similarity(Y, {"jaccard": "jaccard", "only_d": lambda a, b, c, d, n: d / n})

# the original code of Gatto (2021)
labelsim.similarity(Y, ["sorgenfrei.e"], variant="gatto2021")

Y may be a numpy array, a pandas DataFrame (the column names become the label names) or a scipy sparse matrix. Values outside {0, 1} are rejected.

Without the measures argument, all measures of the variant are computed. labelsim.list_measures() lists the names, and labelsim.describe("ochiai.e.2") shows what the catalog records about a name in the two variants.

Divisions by zero and logarithms of zero yield NaN or infinity, with no warning and no replacement by zero. A label that never occurs makes undefined, for example, the phi correlation and the NMI of every pair it takes part in.

scikit-learn

from labelsim.sklearn import LabelSimilarity, make_metric

model = LabelSimilarity(measure="phi").fit(Y)
model.similarity_matrix_

from sklearn.metrics import pairwise_distances
pairwise_distances(Y.T, metric=make_metric("mean_manhattan"))

LabelSimilarity follows the scikit-learn estimator convention and works with clone, get_params and set_params. make_metric returns the value of the measure as it is defined; for pairwise_distances, use a distance-type measure.

Table convention

For the pair (i, j), the library follows Table 1 of Choi, Cha and Tappert (2010).

j present j absent
i present a c
i absent b d

Thus a + c counts the presences of i and a + b those of j, and each formula holds exactly as printed in the source. The matrix returned by similarity uses, at position [i, j], the table of the pair (i, j). Only three of the 65 formulas change when i and j swap places (AMPLE, Tarantula and Peirce); for them, the opposite convention amounts to transposing the matrix.

The Similarity-Measures repository and the text of the thesis use the opposite orientation, with b for i present and j absent. This changes none of the measures used in the thesis, which are symmetric in b and c.

Accepted names

Each measure can be requested by its identifier (ochiai_1), by an alias (ochiai, used in the Similarity-Measures repository), by the original name of Gatto (ochiai.e.1) or by the form with underscores (ochiai_1). In the literature variant, names with the same formula point to the same function. For example, canberra.e, cityblock.e, hamming.e, manhattan.e, minowski.e and square.euclidean.e all yield b + c.

The full list, with the formula, reference and status of each name, is in docs/medidas.md, generated from src/labelsim/catalog/.

How the 107 measures were checked

Each name of Gatto (2021) was compared with the formula of Choi, Cha and Tappert (2010), which gathers 76 measures, and with Table 4 of Kim, Kato and Zhang (2022). The situacao column of catalog/gatto2021.csv records the result.

Status Names
Code equal to the published formula 59
R operator precedence changes the computation (a/a+b equals 1+b) 10
One term replaced by another 9
Other divergence (missing root, discarded numerator) 3
Numerator equal to the denominator, always equals 1 2
No formula checked against an available source 24

The 24 names with no checked source (for example cohen.e, scott.e, loevinger.e and the two versions of Sokal and Sneath IV) exist only in the gatto2021 variant. Requesting them in the literature variant raises MeasureUnavailableError. For Sokal and Sneath IV, eq. 49 of Choi, Cha and Tappert (2010) repeats the term d/(b+d), and the original source was not consulted.

Some formulas of Choi, Cha and Tappert (2010) were kept as printed, even with unusual consequences, and the catalog note records each case. In the cosine (eq. 31) the root appears squared. In Cole (eq. 58), the radicand of the denominator is negative in most tables, which leaves the measure undefined. Fager and McGowan appears in two forms, that of eq. 47 of Choi and that of Kim, Kato and Zhang (2022).

Tests

pytest
  • The gatto2021 variant is compared, on the 107 measures, with the original R code evaluated on 924 contingency tables, with a relative tolerance of 1e-12.
  • The literature variant is compared with the 14 measures of src/measures.R in the Similarity-Measures repository, on the same grid and on the 171 pairs of the birds dataset.
  • The tests also check algebraic identities between the measures (for example Dice = 2J/(1+J) and Hamann = 2SM - 1), the edge cases of identical, complementary, rare and empty labels, and the agreement with scipy.spatial.distance.

The reference values in tests/fixtures/ are generated by experiments/08-python-fixtures.R in the Similarity-Measures repository. The first line of each file records the source commit. After changing the catalog, run python tools/tabela_medidas.py to update docs/medidas.md.

References

CHOI, S.-S.; CHA, S.-H.; TAPPERT, C. C. A survey of binary similarity and distance measures. Journal of Systemics, Cybernetics and Informatics, v. 8, n. 1, p. 43-48, 2010.

GATTO, E. C. MultiLabelSimilaritiesMeasures. GitHub, 2021. Available at https://github.com/cissagatto/MultiLabelSimilaritiesMeasures.

KIM, S.; KATO, I.; ZHANG, X. Comparative analysis of binary similarity measures for compound identification in mass spectrometry-based metabolomics. Metabolites, v. 12, n. 8, 694, 2022.

STREHL, A.; GHOSH, J. Cluster ensembles: a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, v. 3, p. 583-617, 2002.

License

GPL-3.0, the same as the code of Gatto (2021), of which gatto2021.py is a transcription.

Metadata

Release files for labelsim 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for labelsim 0.1.0
File Size Uploaded Provenance Actions
labelsim-0.1.0.tar.gz 42.5 kB Without provenance Details for labelsim-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for labelsim 0.1.0
File Interpreter ABI Platform Actions
labelsim-0.1.0-py3-none-any.whl Python 3 none any Details for labelsim-0.1.0-py3-none-any.whl

Total release size: 78.5 kB

Release files / labelsim-0.1.0.tar.gz

Download URL labelsim-0.1.0.tar.gz
Size 42.5 kB
Tags Source
SHA-256 checksum
How to use checksums
f5c6995c246cf4008a9611e20064e02f9b1f31f23fcda66b478b9d3bd9c7a6ce
BLAKE2b-256 checksum
How to use checksums
3bd8f416ffaff818d4b3348566fa2a984705d68f8be40657051d7e3294a793e3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.9

Release files / labelsim-0.1.0-py3-none-any.whl

Download URL labelsim-0.1.0-py3-none-any.whl
Size 35.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8a71fe8c8e7811cfa78f833c0ae84ded60cc42d2567fabde44e228b0ba5e43df
BLAKE2b-256 checksum
How to use checksums
89c626c18dfe06858ff37a819bff84083d5a0f8eca0a5b00724bf93e4207673c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.9

Security for labelsim 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page