Super K-Means
A super-fast clustering library for high-dimensional vector embeddings
Why Super K-Means?
- Faster clustering of vector embeddings (Cohere, OpenAI, MXBAI, CLIP, MiniLM) than FAISS.
- Index 10M embeddings of 1024 dimensions in less than a minute on a single CPU.
- Faster without compromising clustering quality.
- Support for quantized clustering (8-bit Scalar Quantization, LVQ, and RabitQ)
- Efficient on CPUs (ARM and x86) and GPUs.
Our secret sauce
- Carefully interleaving GEMM routines and pruning kernels that prune dimensions efficiently
- In the benchmarks you see in the cover image, all algorithms are clustering the same data: No dimensionality reduction, no sampling, no early-termination.
Usage
from superkmeans import SuperKMeans
data = ... # Numpy 2D matrix
k = 1000
d = 768
kmeans = SuperKMeans(
n_clusters=k,
dimensionality=d
)
# Run the clustering
centroids = kmeans.train(data) # 2D array with centroids (k x d)
# Get assignments
assignments = kmeans.assign(data)
Then, you can use the centroids to create an IVF index for Vector Search, for example, in FAISS.
Usage in C++
#include <vector>
#include <cstddef>
#include "superkmeans/superkmeans.h"
#include "superkmeans/hierarchical_superkmeans.h"
int main(int argc, char* argv[]) {
std::vector<float> data; // Fill
size_t n = 1000000;
size_t k = 10000;
size_t d = 768;
auto kmeans = skmeans::SuperKMeans(k, d);
// Or Hierarchical Super K-Means for extreme performance:
// auto kmeans = skmeans::HierarchicalSuperKMeans(k, d);
// Run the clustering
std::vector<float> centroids = kmeans.Train(data.data(), n);
// Assign points
std::vector<uint32_t> assignments = kmeans.Assign(data.data(), centroids.data(), n, k);
}
Check our examples for fully working examples in Python and C++.
Quantized Clustering
SuperKMeans supports clustering quantized vectors, substantially accelerating clustering and barely affecting clustering quality. We support 8-bit scalar quantization, LVQ, and RabitQ. You give us float32 vectors and we handle the rest:
kmeans = SuperKMeans(
n_clusters=k,
dimensionality=d,
quantizer='rabitq' # or: sq8, lvq4
)
centroids = kmeans.train(data) # We return float32 centroids
assignments = kmeans.quantized_assign(data, centroids)
[!TIP] Does quantized clustering hurt the index quality? Usually, NO: in most embedding datasets, quantized clustering will get you right where full precision will (+ faster indexing). Read our research paper for the full analysis.
Check our fully working examples in Python or C++.
Documentation
Check our wiki for advanced usage and API reference.
Installation
Python
pip install superkmeans
[!TIP] For maximum performance, we recommend compiling from source.
C++
As a header-only library with CMake FetchContent:
FetchContent_Declare(
superkmeans
GIT_REPOSITORY https://github.com/cwida/superkmeans
)
FetchContent_MakeAvailable(superkmeans)
target_link_libraries(myapp PRIVATE superkmeans)
Compiling Python Bindings from source
Prerequisites
- Clang 17 or GCC 13
- CMake 3.26
- OpenMP
- A BLAS implementation
- Python 3 (only for Python bindings)
git clone https://github.com/cwida/SuperKMeans.git
cd SuperKMeans
git submodule update --init
pip install .
# Run plug-and-play example
python ./examples/simple_clustering.py
# Set a value for n, d and k
python ./examples/simple_clustering.py 200000 1536 1000
Compiling C++ library from source
Prerequisites
- Clang 17 or GCC 13
- CMake 3.26
- OpenMP
- A BLAS implementation
git clone https://github.com/cwida/SuperKMeans.git
cd SuperKMeans
git submodule update --init
# Set proper path to clang if needed
export CXX="/usr/bin/clang++-18"
# Compile
cmake .
make examples
# Run plug-and-play example
cd examples
./simple_clustering.out
# Set a value for n, d and k
./simple_clustering.out 100000 1536 1000
For a more comprehensive installation and compilation guide, check INSTALL.md.
Getting the Best Performance
Check INSTALL.md.
Roadmap
We are actively developing Super K-Means and accepting contributions! Check CONTRIBUTING.md.
Benchmarking
To run our benchmark suite in C++, refer to BENCHMARKING.md.
Adoption
SuperKMeans' ideas have been adopted in:
Other implementations of SuperKMeans
Research behind SuperKMeans
A Super Fast K-means for Indexing Vector Embeddings
@article{kuffo2026super,
title={A Super Fast K-means for Indexing Vector Embeddings},
author={Kuffo, Leonardo and Hepkema, Sven and Boncz, Peter},
journal={arXiv preprint arXiv:2603.20009},
year={2026}
}
Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings
@article{kuffo2026superquantized,
title={Stop Indexing at Full Precision: Revisiting Clustering for Vector Embeddings},
author={Kuffo, Leonardo and Boncz, Peter},
journal={VLDB 2026 Workshop: The 2nd Workshop on Vector Databases},
year={2026}
}
1 maintainer
Release files for superkmeans 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| superkmeans-0.2.0.tar.gz | 52.1 MB | Details |
Built distributions (wheels)
Total release size: 113.7 MB
Release files / superkmeans-0.2.0.tar.gz
| Download URL | superkmeans-0.2.0.tar.gz |
|---|---|
| Size | 52.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b3afd339f13752c183c7caa817d533695fd6ecd57ac8fc114f67e21b1943a84d
|
|
BLAKE2b-256 checksum How to use checksums |
b40bed67af34f0ed106226538fd62c540400f6bb44fdbe5a4e1d6a9a603ace53
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp313-cp313-manylinux_2_28_x86_64.whl
| Download URL | superkmeans-0.2.0-cp313-cp313-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 7.3 MB |
| Tags | CPython 3.13 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
8bfca58b7a1d539f6962cd4032b6a1cfd5aa8753ebe9c86213edbda40615e801
|
|
BLAKE2b-256 checksum How to use checksums |
3c6968aec3280ad43ab4c2329f43693c60228a271f3261a570628c8fc69d9246
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp313-cp313-manylinux_2_28_aarch64.whl
| Download URL | superkmeans-0.2.0-cp313-cp313-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 4.4 MB |
| Tags | CPython 3.13 Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
d6e7c0e35f5c0577a73c022b52138adee4e401095662aba8839522c2bc5d1a3b
|
|
BLAKE2b-256 checksum How to use checksums |
4ada21d58697369c9583f059a5988a470f6f86779e0cebb54c04b685a02600a0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp313-cp313-macosx_11_0_arm64.whl
| Download URL | superkmeans-0.2.0-cp313-cp313-macosx_11_0_arm64.whl |
|---|---|
| Size | 691.7 kB |
| Tags | CPython 3.13 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
0d3c41be52f338521627ff5cc9d274ae2103e1153487e7bffeb7bde9f22891d1
|
|
BLAKE2b-256 checksum How to use checksums |
8ccca697e50dee2e7d689357887307cdf2f4abcbae83e82e8bf7e2530f8c5cae
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp312-cp312-manylinux_2_28_x86_64.whl
| Download URL | superkmeans-0.2.0-cp312-cp312-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 7.3 MB |
| Tags | CPython 3.12 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
eac9cc411f2fb280e484d164b88aa1f5540bddb93d90a3fe24fc6860cbf5c7ca
|
|
BLAKE2b-256 checksum How to use checksums |
83a55f53951fee12a2d12d6c05f1adde556278cb097248dc74f8c7e223c02228
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp312-cp312-manylinux_2_28_aarch64.whl
| Download URL | superkmeans-0.2.0-cp312-cp312-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 4.4 MB |
| Tags | CPython 3.12 Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
7344aa398cfab6d4d7487b6c5ade987f39885aa32e9d27405788189b1d1019dd
|
|
BLAKE2b-256 checksum How to use checksums |
5aa25d9c46ffdf1ecd216ad7c941742d4fed2b8b905c589fc03846e966b08119
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp312-cp312-macosx_11_0_arm64.whl
| Download URL | superkmeans-0.2.0-cp312-cp312-macosx_11_0_arm64.whl |
|---|---|
| Size | 691.6 kB |
| Tags | CPython 3.12 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
ec0f6cadb0bbe8022a4cd6bde723d393a92264a2983e1fb3e0d25c174069d572
|
|
BLAKE2b-256 checksum How to use checksums |
883e5f7ec6f81c31f05707f27b0435aa180c0f1c4dbed9188810c9547ecfcdaf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp311-cp311-manylinux_2_28_x86_64.whl
| Download URL | superkmeans-0.2.0-cp311-cp311-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 7.3 MB |
| Tags | CPython 3.11 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
6faa60fc4eb455a0d264c0c735835aa22642a0920f87e5dd3139ae8a68021998
|
|
BLAKE2b-256 checksum How to use checksums |
35abcf6ad4bb902227eeae18176b54f5453d7a4e9a1f44e765fd6749770b7611
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp311-cp311-manylinux_2_28_aarch64.whl
| Download URL | superkmeans-0.2.0-cp311-cp311-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 4.4 MB |
| Tags | CPython 3.11 Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
2bdd682ff854725e9e3316165365587e6ce3169d01b2a6d8d35bc53d98b2b6dc
|
|
BLAKE2b-256 checksum How to use checksums |
1b1c2f0bcf2b56bef4519249cefd1cdfaac84604f8674d7a54c54cabffd51353
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp311-cp311-macosx_11_0_arm64.whl
| Download URL | superkmeans-0.2.0-cp311-cp311-macosx_11_0_arm64.whl |
|---|---|
| Size | 689.4 kB |
| Tags | CPython 3.11 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
526261ffb5cc59b93528b25bc54c73955e0cab72006128752c5e467f6099f315
|
|
BLAKE2b-256 checksum How to use checksums |
cd7b170557cf6380b8952ff6ca7e1caab06cf5f1dc5edfa95b71cfa75425c795
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp310-cp310-manylinux_2_28_x86_64.whl
| Download URL | superkmeans-0.2.0-cp310-cp310-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 7.3 MB |
| Tags | CPython 3.10 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
2c73757e74cddc64ed09d6370200b30603ab5ef6399f1f34a9892290eb58141e
|
|
BLAKE2b-256 checksum How to use checksums |
bb1c308d0d3d73eb8e61dc14db726ed89abd46b8f696fd5775cb943447002424
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp310-cp310-manylinux_2_28_aarch64.whl
| Download URL | superkmeans-0.2.0-cp310-cp310-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 4.4 MB |
| Tags | CPython 3.10 Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
2f11b96e22517d1d00cc1d92a0ac93af608a5d694a7fe908df81b4c80c76b174
|
|
BLAKE2b-256 checksum How to use checksums |
2890ef0c2d543a7163fe1e07526467bcea849ee7a25af2d92e810b5eb30086bb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp310-cp310-macosx_11_0_arm64.whl
| Download URL | superkmeans-0.2.0-cp310-cp310-macosx_11_0_arm64.whl |
|---|---|
| Size | 688.1 kB |
| Tags | CPython 3.10 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
3c9ac5b1c81d9b1da33318abb8185cebb8795453e855339b65d6610bd4972003
|
|
BLAKE2b-256 checksum How to use checksums |
07426284a68a6ef7a34b31ce0b6d6cc8d80df10f1027f8354532a264433dc808
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp39-cp39-manylinux_2_28_x86_64.whl
| Download URL | superkmeans-0.2.0-cp39-cp39-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 7.3 MB |
| Tags | CPython 3.9 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
5fd95393404f49d6fb70ff7cfe92819f996aa250fa1e28a002d2d162ec223a8b
|
|
BLAKE2b-256 checksum How to use checksums |
24f46f0e105f948d5f95aa02b9db8572cd71804abbcdcc90fdcdd2c73cdb7219
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp39-cp39-manylinux_2_28_aarch64.whl
| Download URL | superkmeans-0.2.0-cp39-cp39-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 4.4 MB |
| Tags | CPython 3.9 Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
f7ec692da4ff3e8f2dc366fddf2bd39116f1cb0a0447235c581b333b8ba3015a
|
|
BLAKE2b-256 checksum How to use checksums |
dd9654db8a3966aa6b8ba0db7b1738d599f6afd3f5f59b4c0b73e2ce9b4b151b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency logRelease files / superkmeans-0.2.0-cp39-cp39-macosx_11_0_arm64.whl
| Download URL | superkmeans-0.2.0-cp39-cp39-macosx_11_0_arm64.whl |
|---|---|
| Size | 688.2 kB |
| Tags | CPython 3.9 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
b59a5163411dfba6d3c58680923e65e47f8098c67154b7309fe99ea50578c5e6
|
|
BLAKE2b-256 checksum How to use checksums |
6cad85a22b63faefbd98f7d743fd73c5b74d82de531ee6f3357fa0942d470870
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Aug 28, 2026.
Transparency log
