sentence-embedder

A lightweight internal Python library for generating sentence/semantic embeddings with a clean, unified API.

Features

Installation

# Clone / copy this package into your project, then:

# SentenceTransformers backend (local, recommended)
pip install -e ".[st]"

# OpenAI backend
pip install -e ".[openai]"

# Both
pip install -e ".[all]"

# With dev tools (pytest, ruff, mypy)
pip install -e ".[all,dev]"

Quick Start

1 — Basic embedding

from sentence_embedder import SentenceEmbedder

embedder = SentenceEmbedder()                 # defaults: SentenceTransformers, all-MiniLM-L6-v2

vec = embedder.embed("The cat sat on the mat.")
print(vec.shape)   # (384,)

2 — Batch embedding

texts = ["Hello world", "Machine learning is fun", "I love Python"]
vecs = embedder.embed_batch(texts)
print(vecs.shape)  # (3, 384)

3 — Cosine similarity

score = embedder.similarity("fast car", "quick automobile")
print(score)  # ~0.85

4 — Semantic search

corpus = [
    "The stock market crashed today.",
    "Scientists discover a new planet.",
    "Football team wins the championship.",
    "A new AI model beats human performance.",
]

results = embedder.most_similar("breakthrough in artificial intelligence", corpus, top_k=2)
for sentence, score in results:
    print(f"{score:.3f}  {sentence}")

5 — Disk cache (avoid re-embedding)

from sentence_embedder import SentenceEmbedder, EmbeddingCache

base = SentenceEmbedder()
embedder = EmbeddingCache(base, cache_dir=".cache/embeddings")

vec = embedder.embed("Hello world")   # computed → stored on disk
vec = embedder.embed("Hello world")   # loaded from cache instantly

OpenAI Backend

from sentence_embedder import SentenceEmbedder

embedder = SentenceEmbedder(
    backend="openai",
    model_name="text-embedding-3-small",
    openai_api_key="sk-...",   # or set OPENAI_API_KEY env var
)

vec = embedder.embed("Hello from OpenAI!")

API Reference

SentenceEmbedder

EmbeddingCache

Running Tests

pytest
# With coverage:
pytest --cov=sentence_embedder --cov-report=term-missing

Package Structure

sentence_embedder/
├── sentence_embedder/
│   ├── __init__.py       # Public exports
│   ├── embedder.py       # SentenceEmbedder (core)
│   └── cache.py          # EmbeddingCache (disk cache)
├── tests/
│   └── test_embedder.py  # Unit tests (no model/network needed)
├── pyproject.toml        # Build config & dependencies
└── README.md

Choosing a Model