Skip to main content

embedsim

Measure semantic similarity and detect outliers in text collections using embeddings.

embedsim is a lightweight Python library that helps you understand how well texts relate to each other. It provides two core functions: pairwise similarity for comparing two texts, and group coherence for analyzing collections.

Use cases:

  • Content moderation: Find off-topic comments or reviews
  • Document clustering: Identify outliers before grouping
  • Quality assurance: Verify generated content stays on topic
  • Search relevance: Score how well results match a query theme
  • Duplicate detection: Compare documents for similarity

Installation

For OpenAI models:

uv add embedsim[openai]
export OPENAI_API_KEY=your-key-here

For local models:

uv add embedsim[sentence-transformers]

Quick Start

Pairwise Similarity

Compare two texts directly:

import embedsim

# Similar texts
score = embedsim.pairsim(
    "The cat sat on the mat",
    "A feline rested on the rug"
)
print(score)  # 0.89

# Dissimilar texts
score = embedsim.pairsim(
    "The cat sat on the mat",
    "Python is a programming language"
)
print(score)  # 0.21

Group Coherence

Analyze a collection and find outliers:

import embedsim

texts = [
    "Python is a programming language",
    "JavaScript is used for web development",
    "Machine learning uses neural networks",
    "Pizza is a popular food"  # This doesn't belong
]

scores = embedsim.groupsim(texts)
# [0.76, 0.73, 0.71, 0.28]
#                    ~~~~ Outlier detected!

API Reference

pairsim(text_a, text_b, model_id=None, **config) → float

Compute similarity between two texts.

  • Converts both texts to embeddings
  • Computes cosine similarity
  • Returns a single similarity score (0-1, higher = more similar)

groupsim(texts, model_id=None, **config) → list[float]

Compute coherence scores for a collection of texts.

  • Converts all texts to embeddings
  • Calculates the centroid (average) of all embeddings
  • Measures how close each text is to the centroid
  • Returns coherence scores (0-1, higher = more coherent)

This centroid-based approach gives you a score per text showing how well it fits with the group's semantic theme.

Configuration

Runtime Configuration

Modify the config object directly in your code:

import embedsim

# Change default model at runtime
embedsim.config.model = "jinaai/jina-embeddings-v2-base-en"

# Now all calls use the new default
score = embedsim.pairsim("hello", "hi")

Environment Variables

Alternatively, set configuration via environment variables:

# Set default model
export EMBEDSIM_MODEL=jinaai/jina-embeddings-v2-base-en

# Use custom OpenAI key
export EMBEDSIM_OPENAI_API_KEY=sk-...

Models

embedsim supports both OpenAI's API and local sentence-transformer models.

See MODELS.md for detailed model comparison and selection guide.

OpenAI (default, requires API key):

# Best for production - fast, accurate, no model downloads
score = embedsim.pairsim(text_a, text_b)  # uses openai/text-embedding-3-small
scores = embedsim.groupsim(texts, model_id="openai/text-embedding-3-large")

Local models (privacy, offline):

# Run entirely on your machine
score = embedsim.pairsim(text_a, text_b, model_id="jinaai/jina-embeddings-v2-base-en")
scores = embedsim.groupsim(texts, model_id="sentence-transformers/all-MiniLM-L6-v2")

Development

# Install with dev dependencies
uv sync --all-extras

# Run tests and benchmarks
make test

License

MIT

Metadata

Release files for embedsim 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for embedsim 0.1.1
File Size Uploaded Provenance Actions
embedsim-0.1.1.tar.gz 9.7 kB Without provenance Details for embedsim-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for embedsim 0.1.1
File Interpreter ABI Platform Actions
embedsim-0.1.1-py3-none-any.whl Python 3 none any Details for embedsim-0.1.1-py3-none-any.whl

Total release size: 17.4 kB

Release files / embedsim-0.1.1.tar.gz

Download URL embedsim-0.1.1.tar.gz
Size 9.7 kB
Tags Source
SHA-256 checksum
How to use checksums
e6849065fad3e644c4d1962b71f725a562a7cec1395582e317caad1e9138e1b4
BLAKE2b-256 checksum
How to use checksums
058493bf9c6986891bf76bfaf9739c8cb6108f7e81ca97d959385e3ebf227acd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.23

Release files / embedsim-0.1.1-py3-none-any.whl

Download URL embedsim-0.1.1-py3-none-any.whl
Size 7.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7eb429d7a0663d998481d30f790c260b1c54ccc725d41ebc997a2d9aca6f9a12
BLAKE2b-256 checksum
How to use checksums
de67157c07050d59ededf42d5027309282e240a8d30e22acac86cb50a9cdf3d3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.23

Security for embedsim 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page