Skip to main content

Fast ABX evaluation

PyPI Python CI Documentation License

fastabx is a Python package for efficient computation of ABX discriminability.

The ABX discriminability measures how well categories of interest are separated in the representation space by determining whether tokens from the same category are closer to each other than to those from a different category. While ABX has been mostly used to evaluate speech representations, it is a generic framework that can be applied to other domains of representation learning.

This package provides a simple interface that can be adapted to any ABX conditions, and to any input modality.

  • Generic: any ON, BY and ACROSS conditions, on dense or discrete representations, in any modality.
  • Fast: triplets are built with lazy polars queries, and distances are computed in batch on CPU or GPU, with DTW running as a PyTorch C++/CUDA extension.
  • Ready to use: zerospeech_abx and the fastabx command line interface reproduce the ZeroSpeech triphone and phoneme ABX out of the box.

Check out the documentation for more information: https://docs.cognitive-ml.fr/fastabx

Install

Install the pre-built package in your environment:

pip install fastabx

It requires Python 3.12 or later, and depends on PyTorch 2.10.0 or later, NumPy, Polars, tqdm, and torchdtw.

Quickstart

Simple example:

import numpy as np
from fastabx import Dataset, Score, Task

rng = np.random.default_rng(0)
features = np.concatenate([rng.normal(0, 1, (50, 8)), rng.normal(2, 1, (50, 8))])
labels = {"phone": ["a"] * 50 + ["b"] * 50, "speaker": ["s1", "s2"] * 50}

dataset = Dataset.from_numpy(features, labels)  # What to compare
task = Task(dataset, on="phone", by=["speaker"])  # Which triplets to build
score = Score(task, "euclidean")  # How to compare them

print(score.collapse(levels=["speaker"]))  # ABX error rate
# 0.03059999999999999

On speech, build the dataset from an item file and a directory of features instead, then run the same Task / Score pipeline:

from fastabx import Dataset

dataset = Dataset.from_item("./triphone-dev-clean.item", "./hubert-l11-dev-clean", frequency=50)
task = Task(dataset, on="#phone", by=["speaker", "next-phone", "prev-phone"])
score = Score(task, "angular")
print(score.collapse(levels=[("next-phone", "prev-phone"), "speaker"]))

The standard ZeroSpeech evaluation is available as a single function, and as a CLI:

from fastabx import zerospeech_abx

error_rate = zerospeech_abx(
    "./triphone-dev-clean.item",
    "./hubert-l11-dev-clean",
    max_size_group=10,
    speaker="within",
    context="within",
)
fastabx ./triphone-dev-clean.item ./hubert-l11-dev-clean --max-size-group 10
# ABX error rate: 3.378%

Scores are ABX error rates: lower is better and chance is 0.5.

See the user guide for the full pipeline, the API reference for every option, and the examples for subsampling, pooling and constraints.

Citation

A preprint is available on arXiv: https://arxiv.org/abs/2505.02692
If you use fastabx in your work, please cite it:

@misc{fastabx,
  title={fastabx: A library for efficient computation of ABX discriminability},
  author={Maxime Poli and Emmanuel Chemla and Emmanuel Dupoux},
  year={2025},
  eprint={2505.02692},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2505.02692},
}

License

fastabx is released under the MIT license.

Metadata

Release files for fastabx 0.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fastabx 0.10.0
File Size Uploaded Provenance Actions
fastabx-0.10.0.tar.gz 485.7 kB With provenance Details for fastabx-0.10.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for fastabx 0.10.0
File Interpreter ABI Platform Actions
fastabx-0.10.0-py3-none-any.whl Python 3 none any Details for fastabx-0.10.0-py3-none-any.whl

Total release size: 531.8 kB

Release files / fastabx-0.10.0.tar.gz

Download URL fastabx-0.10.0.tar.gz
Size 485.7 kB
Tags Source
SHA-256 checksum
How to use checksums
495fb2751f41162e707264d1c459f8bc21c758dd6d9972c86582a3715609c4fe
BLAKE2b-256 checksum
How to use checksums
e7f1be8cb984a5e1c91c4080d8a266019562c8b40e9751f984b9b9e7af2fa0a3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 6, 2026.

Transparency log

Release files / fastabx-0.10.0-py3-none-any.whl

Download URL fastabx-0.10.0-py3-none-any.whl
Size 46.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
116ba6ad898387d127f7bb08b878bd99334469a783f09b9c4f20bab53b5b4b58
BLAKE2b-256 checksum
How to use checksums
a5a208609b5baccbbfff20bea8e1dd959e7050c8633ce0a1ca09ba50b96ef95b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 6, 2026.

Transparency log

Security for fastabx 0.10.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for fastabx-0.10.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 6, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page