String Grouper

Click to see image

The image displayed above is a visualization of the graph-structure of one of the groups of strings found by string_grouper. Each circle (node) represents a string, and each connecting arc (edge) represents a match between a pair of strings with a similarity score above a given threshold score (here 0.8).

The centroid of the group, as determined by string_grouper (see tutorials/group_representatives.md for an explanation), is the largest node, also with the most edges originating from it. A thick line in the image denotes a strong similarity between the nodes at its ends, while a faint thin line denotes weak similarity.

The power of string_grouper is discernible from this image: in large datasets, string_grouper is often able to resolve indirect associations between strings even when, say, due to memory-resource-limitations, direct matches between those strings cannot be computed using conventional methods with a lower threshold similarity score.

———

^{This image was designed using the graph-visualization software Gephi 0.9.2 with data generated by string_grouper operating on the sec__edgar_company_info.csv sample data file.}

---

string_grouper is a library that makes finding groups of similar strings within a single, or multiple, lists of strings easy — and fast. string_grouper uses tf-idf to calculate cosine similarities within a single list or between two lists of strings. The full process is described in the blog Super Fast String Matching in Python.

Installing

pip install string-grouper

Speed

string_grouper leverages the blazingly fast sp_matmul_rs (originally based on: sparse_dot_topn) to calculate cosine similarities.

s = datetime.datetime.now()
matches = match_strings(names["name"], number_of_processes=15)
e = datetime.datetime.now()

diff = e - s
print(diff)

Results in:

00:17.80 On an m5 pro, where len(names) = 663 000

in other words, the library is able to perform fuzzy matching of 663 000 names in less than 18 seconds on a 2026 consumer CPU using 15 cores.

Simple Match

import pandas as pd
from string_grouper import match_strings

company_names = 'sec__edgar_company_info.csv'
companies = pd.read_csv(company_names)
# Create all matches:
matches = match_strings(companies['Company Name'])
# Look at only the non-exact matches:
matches[matches['left_Company Name'] != matches['right_Company Name']].head()

Group Similar Strings and Find most Common

companies[["group-id", "name_deduped"]] = group_similar_strings(companies['Company Name'])
companies.groupby('name_deduped')['Line Number'].count().sort_values(ascending=False).head(10)

Documentation

The documentation can be found here

Backends

The library was originally developed using the sparse_dot_topn library, but has since been rewritten to use the sp_matmul_rs library, which is a Rust implementation of the sparse matrix multiplication algorithm optimized with Claude Fable. To run the library with the original sparse_dot_topn backend, set use_sp_matmul_rs to False.