Skip to main content
EVōC Logo

EVōC

EVōC (pronounced as “evoke”) provides Embedding Vector Oriented Clustering. EVōC is a library for fast and flexible clustering of large datasets of high dimensional embedding vectors. If you have CLIP-vectors, outputs from sentence-transformers, or openAI, or Cohere embed, and you want to quickly get good clusters out this is the library for you. EVōC takes all the good parts of the combination of UMAP + HDBSCAN for embedding clustering, improves upon them, and removes all the time-consuming parts. By specializing directly to embedding vectors we can get good quality clustering with fewer hyper-parameters to tune and in a fraction of the time.

EVōC is the library to use if you want:

  • Fast clustering of embedding vectors on CPU

  • Multi-granularity clustering, and automatic selection of the number of clusters

  • Clustering of int8 or binary quantized embedding vectors that works out-of-the-box

As of now this is very much an early beta version of the library. Things can and will break right now. We would welcome feedback, use cases and feature suggestions however.

Basic Usage

EVōC follows the scikit-learn API, so it should be familiar to most users. You can use EVōC wherever you might have previously been using other sklearn clustering algorithms. Here is a simple example

import evoc
from sklearn.datasets import make_blobs

data, _ = make_blobs(n_samples=100_000, n_features=1024, centers=100)

clusterer = evoc.EVoC()
cluster_labels = clusterer.fit_predict(data)

Some more unique features include the generation of multiple layers of cluster granularity, the ability to extract a hierarchy of clusters across those layers, and automatic duplicate (or very near duplicate) detection.

import evoc
from sklearn.datasets import make_blobs

data, _ = make_blobs(n_samples=100_000, n_features=1024, centers=100)

clusterer = evoc.EVoC()
cluster_labels = clusterer.fit_predict(data)
cluster_layers = clusterer.cluster_layers_
hierarchy = clusterer.cluster_tree_
potential_duplicates = clusterer.duplicates_

The cluster layers are a list of cluster label vectors with the first being the finest grained and later layers being coarser grained. This is ideal for layered topic modelling and use with DataMapPlot. See this data map for an example of using these layered clusters in topic modelling (zoom in to access finer grained topics).

Installation

EVōC has a small set of dependencies:

  • numpy

  • scikit-learn

  • numba

  • tqdm

  • tbb

You can install EVōC from PyPI using pip:

pip install evoc

To install the latest version of EVōC from source:

pip install git+https://github.com/TutteInstitute/evoc.git

License

EVōC is BSD (2-clause) licensed. See the LICENSE file for details.

Contributing

Contributions are more than welcome! If you have ideas for features of projects please get in touch. Everything from code to notebooks to examples and documentation are all equally valuable so please don’t feel you can’t contribute. To contribute please fork the project make your changes and submit a pull request. We will do our best to work through any issues with you and get your code merged in.

Metadata

Release files for evoc 0.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for evoc 0.3.1
File Size Uploaded
evoc-0.3.1.tar.gz 55.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for evoc 0.3.1
File Interpreter ABI Platform
evoc-0.3.1-py3-none-any.whl Python 3 none any Details

Total release size: 116.5 kB

Release files / evoc-0.3.1.tar.gz

Download URL evoc-0.3.1.tar.gz
Size 55.9 kB
Tags Source
SHA-256 checksum
How to use checksums
e4986174419885479929595eea471593e06afee8a3abd8d6934e1ca7530ec449
BLAKE2b-256 checksum
How to use checksums
6e16f5309a9137e7669489cb9b53a1e52f9ee0f38ae794c2f1a7174fa547d189
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.10.20

Release files / evoc-0.3.1-py3-none-any.whl

Download URL evoc-0.3.1-py3-none-any.whl
Size 60.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
28e4d73d3e991821a722ba8fb0913dd38e1bb52a26a923caf5fc1c218115161e
BLAKE2b-256 checksum
How to use checksums
81519fae6b38d7cd38b8b030d612943941b4018ddca83c9d6a11731c0dae71cc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.10.20

Release history Release notifications | RSS feed

This release

0.3.1 This release

2 release files

0.1.3

2 release files

0.1.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page