This release is a pre-release and may not be stable for production use.
everalgo-clustering
Online incremental clustering for EverAlgo — cluster_by_geometry (sync) and acluster_by_llm (async, LLM-refined; cluster_by_llm is its sync bridge) operating on caller-owned list[Cluster] state.
Stateless: the package never embeds, never queries storage, never holds a lock. The caller owns embedding computation, persistence (list[Cluster] serialisation), and read-modify-write coordination across concurrent writers.
See the umbrella project: EverAlgo monorepo and the architecture document at docs/concepts/architecture.md.
Install
pip install everalgo-clustering
What this distribution provides
| Symbol | Role |
|---|---|
Cluster |
Frozen Pydantic value object — one cluster snapshot. Caller-supplied id / members; algorithm supplies merged centroid, count, last_ts, preview. |
cluster_by_geometry |
Cosine similarity + time-window filter + threshold; no LLM; sync |
acluster_by_llm |
Top-K geometric recall → fast-path skip → LLM semantic ranking; raises on LLM failure; async |
cluster_by_llm |
Blocking sync bridge over acluster_by_llm; only outside a running event loop |
Cluster type
class Cluster(BaseModel):
id: str | None = None # caller-supplied business id; algo only passes through, never mints
centroid: np.ndarray
count: int = 1
last_ts: int # Unix epoch milliseconds
preview: list[str] = []
members: list[str] = [] # caller-supplied entity ids; algo appends on merge, never inspects
The caller wraps each incoming item as a size-1 Cluster (count=1) before passing to either function.
Both functions return Cluster | None: a merged snapshot when the item is assigned to an existing cluster, or None when no match — the caller then appends the original size-1 Cluster as a brand-new entry and mints its own id.
Quick start
import numpy as np
from everalgo.clustering import Cluster, cluster_by_geometry
existing: list[Cluster] = [] # caller loads from storage; empty on first run
vector = np.random.rand(2560).astype(np.float32)
timestamp_ms = 1_700_000_000_000
new_cluster = Cluster(centroid=vector, last_ts=timestamp_ms)
merged = cluster_by_geometry( # sync — no await
new_cluster,
existing,
threshold=0.65, # cosine similarity floor
time_window_days=7.0, # ignore clusters older than this window
)
if merged is not None:
# item assigned to an existing cluster; caller updates the matching entry
print(f"merged into cluster id={merged.id!r}, new count={merged.count}")
else:
# no match — caller appends new_cluster and stamps its own id
new_cluster_with_id = new_cluster.model_copy(update={"id": "cid_001"})
existing.append(new_cluster_with_id)
print("created new cluster")
LLM-refined clustering
acluster_by_llm adds a semantic ranking step over the top-K geometrically-nearest candidates. It raises on LLM failure — there is no internal fallback and no retry; the caller decides whether to retry or fall back to cluster_by_geometry. The call asks for a strict {"idx": int, "reason": str} object (idx is the candidate's index in existing_clusters, -1 for a new cluster); a reply that is not exactly that raises everalgo.llm.LLMOutputError. The call sends a system message of rules and a user message holding the candidate clusters and the new item's preview. custom_instructions= adds the host's text to the rules, never replacing them: it is placed verbatim, never rendered as a template, in a <custom_instructions> block that is always the system message's last, and None or a blank string leaves it out. There is no prompt=: the bundled rules cannot be replaced per call.
Populate new_cluster.preview with the item's representative text so the LLM has something to rank against.
import asyncio
import numpy as np
from everalgo.clustering import Cluster, acluster_by_llm
from everalgo.testing import FakeLLMClient
_LLM_JSON = '{"idx": 0, "reason": "same domain"}'
async def main() -> None:
fake = FakeLLMClient(responses=[_LLM_JSON])
existing: list[Cluster] = []
vector = np.random.rand(2560).astype(np.float32)
new_cluster = Cluster(
centroid=vector,
last_ts=1_700_000_000_000,
preview=["Python async retry patterns"], # shown to the LLM
)
merged = await acluster_by_llm(
new_cluster,
existing,
llm=fake,
k_candidates=30,
llm_skip_threshold=0.85,
)
print(f"merged: {merged}")
asyncio.run(main())
Persistence pattern
Cluster is a frozen Pydantic model — serialise with model_dump() and reconstruct with Cluster.model_validate(). The caller owns the list and the lock:
raw_list = await store.load(user_id) or []
clusters = [Cluster.model_validate(r) for r in raw_list]
async with caller.lock(f"cluster:{user_id}"):
merged = cluster_by_geometry(new_cluster, clusters)
if merged is not None:
idx = next(i for i, c in enumerate(clusters) if c.id == merged.id)
clusters[idx] = merged
else:
new_cluster_stamped = new_cluster.model_copy(update={"id": new_id})
clusters.append(new_cluster_stamped)
await store.save(user_id, [c.model_dump() for c in clusters])
API reference
def cluster_by_geometry(
new_cluster: Cluster,
existing_clusters: list[Cluster],
*,
threshold: float = 0.65,
time_window_days: float = 7.0,
preview_cap: int = 5,
) -> Cluster | None: ...
async def acluster_by_llm(
new_cluster: Cluster,
existing_clusters: list[Cluster],
*,
llm: LLMClient,
k_candidates: int = 30,
llm_skip_threshold: float = 0.85,
custom_instructions: str | None = None,
preview_cap: int = 5,
) -> Cluster | None: ...
cluster_by_llm = async_to_sync(acluster_by_llm) # same signature, blocking
Both functions return the merged Cluster (existing cluster's id preserved, centroid/count/members updated) or None (no match — caller creates a new cluster entry and mints its own id).
Tested embedding model: Qwen3-Embedding-4B (2560-dim float32). Any consistent-dimension embedding works; EverAlgo does not import or manage embedding SDKs.
Related distributions
everalgo-user-memory— usescluster_by_geometryin the full user-memory pipelineeveralgo-agent-memory— usesacluster_by_llmfor skill clustering
Metadata
Release files for everalgo-clustering 0.3.0rc1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| everalgo_clustering-0.3.0rc1.tar.gz | 19.0 kB | Without provenance | Details for everalgo_clustering-0.3.0rc1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| everalgo_clustering-0.3.0rc1-py3-none-any.whl | Python 3 | none | any | Details for everalgo_clustering-0.3.0rc1-py3-none-any.whl |
Total release size: 33.5 kB
Release files / everalgo_clustering-0.3.0rc1.tar.gz
| Download URL | everalgo_clustering-0.3.0rc1.tar.gz |
|---|---|
| Size | 19.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
abe321dfefb80631f13edd8215c6d28525f5dc5339d4f2a946cc6c27355d747e
|
|
BLAKE2b-256 checksum How to use checksums |
27b3fa870a115f3e771a7a307c3d22121daa202d6f2098f3648f29de44502e8d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.12.12
|
Release files / everalgo_clustering-0.3.0rc1-py3-none-any.whl
| Download URL | everalgo_clustering-0.3.0rc1-py3-none-any.whl |
|---|---|
| Size | 14.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
6806969f12724e69945c29886b25816a3221023755b83c9e9efd7f396f85e501
|
|
BLAKE2b-256 checksum How to use checksums |
2dfa301f31517552edf57998b6d940707b607ae1f40a6844dc8da27cf2f7d5f4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.12.12
|
Security for everalgo-clustering 0.3.0rc1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.