bloom-torch 1.0.0

Bloom-filter–accelerated clustering and set structures in PyTorch (not related to the BLOOM language model).

bloom-torch

bloom-torch is a small PyTorch library for Bloom filters, Bloom matrices (bitwise set encodings), and BloomKMeans — K-means with Bloom-based candidate pruning when the number of clusters is large.

Note: This package is not related to the BLOOM multilingual language model or other “BLOOM” model names in the Hugging Face ecosystem. The name refers to Bloom filters (probabilistic set membership, 1970).

Install

pip install bloom-torch

(After the first release; for now, from a clone: pip install -e .)

Requirements

• Python ≥ 3.10
• PyTorch ≥ 2.0

Quick start (v0.1)

import torch
from bloom_torch import BloomKMeans

# X: [n, d] float32 — e.g. token or item embeddings (cluster on raw or normalised data as you prefer)
X = torch.randn(10_000, 128, dtype=torch.float32)

km = BloomKMeans(n_clusters=256, topk_cache=16, bm_fp_rate=0.01, routing_fp_rate=0.01, seed=0)
km.fit(X, max_iters=20, use_bm_after=1, allow_bm_assign_small_k=False)

# Optional: build a cluster → element Bloom matrix for routing / masking
routing = km.build_routing_bloom(vocab_size=X.shape[0])

Public API in v0.1: BloomHasher, TorchBloomMatrix, BloomKMeans.
Routing hooks, logits processors, and LLM-specific helpers may be added in later versions or live in application code.

Relationship to research code

This repository was split out from the PyBloomFilter research prototype (torch_bloom package) so the core tensor algorithms can be versioned and published independently.

License

MIT — see LICENSE.