lens-embed 0.1.1

Shared embedding helper for the lens family — one pinned text + image model so every member's vectors are comparable (for cross-artefact consistency and cohort distinctiveness).

lens-embed

Part of the lens family.

The family's shared embedding helper. One pinned text model and one pinned image model, so vectors produced by every member land in the same space and are directly comparable.

That comparability is the whole point. It's what makes two family-level signals possible:

• Cross-artefact consistency — does the video narration match the report? does the code match what the reflection claims?
• Cohort distinctiveness — is this submission an outlier, or unusually similar to another in the same batch? (Surfaced as a neutral observation for a human to interpret — never a verdict.)

lens-embed is a library (role: library, like lens-contract), not an analyser. It produces vectors and compares them; it makes no judgements.

Producer vs consumer

The heavy ML backends are opt-in, because the two sides need different things:

You are…	You install	You get
a producer (an analyser generating vectors)	lens-embed[text] and/or lens-embed[image]	embed_text / embed_image
a consumer (comparing pre-computed vectors)	lens-embed (numpy only)	cosine_similarity, pairwise_similarity, most_similar

pip install 'lens-embed[text]'    # sentence-transformers (all-MiniLM-L6-v2)
pip install 'lens-embed[image]'   # CLIP via open_clip (ViT-B-32)
pip install 'lens-embed[all]'     # both
pip install lens-embed            # comparison helpers only (numpy)

Usage

from lens_embed import embed_text, cosine_similarity, most_similar

a = embed_text("Climate disclosure in annual reports")
b = embed_text("How companies report on the environment")
cosine_similarity(a, b)            # ~0.6 — L2-normalised, so this is a dot product

# A whole document → one pooled vector (use this for doc-level comparison):
from lens_embed import embed_long_text
doc_vec = embed_long_text(full_report_text)

# Rank a cohort against one submission (distinctiveness):
cohort = [embed_long_text(s) for s in submissions]
most_similar(cohort[0], cohort[1:], top_k=3)   # [(idx, score), ...]

from lens_embed import embed_image
vec = embed_image("diagram.png")   # CLIP image vector

Degrade gracefully when a backend isn't installed:

from lens_embed import backend_available, embed_text
emb = embed_text(text) if backend_available("text") else None

Model identity (the contract)

Modality	Default	Why
text	all-MiniLM-L6-v2	the model conversation-/document-analyser already load — adopting lens-embed doesn't move the existing vector space
image	ViT-B-32 / laion2b_s34b_b79k (CLIP)	joint image/text space for visual comparison

The defaults are the contract — don't diverge per analyser, or vectors stop being comparable. They're env-overridable (LENS_EMBED_TEXT_MODEL, etc.) for experiments only.

Note: sentence-transformers text vectors and CLIP text vectors live in different spaces. Use embed_text for text-to-text comparison; CLIP (embed_image) for anything visual.

Development

uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"
pytest                  # fast: comparison + contract (numpy only)
pytest -m slow          # loads real models (needs [text] / [image])

License

MIT — see LICENSE.