mantissa-embed

Learning what "similar" means, with a C engine. A classifier answers which class; an embedder answers how alike — it maps each image to a point in a vector space where same-thing images land close together and different-thing images land far apart. That single geometry powers similarity search, verification ("are these two the same?") and retrieval ("show me the nearest matches"), for classes the model was never trained to name.

mantissa-embed trains that embedding with metric learning — the contrastive and triplet losses — on top of a mantissa-cnn trunk: its Conv2D / MaxPool2D / Flatten / Dense layers, its mantissa C-engine and pure-numpy backends, and its dataset loaders are reused, not reimplemented. This package adds only what metric learning needs on top of a classifier: the two losses, an Embedder that trains a trunk with them, and the four things embeddings buy you — fit / embed / retrieve / verify.

The mantissa family

Part of the mantissa family: a low-precision engine written in C, with small Python packages built on top. Each package sits under the one it depends on — ⭐ marks where you are, and every other name links to its repo.

• mantissa — low-precision neural-network engine in C (the core)
  • mantissa-perceptron — perceptron & ADALINE, the linear classics
  • mantissa-nn — shared neural-net primitives (layers, engine binding)
    • mantissa-cnn — convolutional networks for images
      • mantissa-auto-encoder — autoencoders for denoising & super-resolution
      • mantissa-interpret — CNN interpretability (occlusion, saliency, Grad-CAM)
      • ⭐ mantissa-embed — CNN metric learning (image embeddings for similarity & retrieval) (you are here)
    • mantissa-mlp — multilayer perceptrons, fully-connected nets

New to metric learning?

A classifier learns a fixed set of labels and a decision boundary between them. Metric learning learns something more basic and more reusable: a distance. It trains the network so that the Euclidean distance between two images' embeddings is a measure of how similar they are — and it does that using only the relation between examples ("these two are the same kind", "these two are not"), never a class name. Two consequences follow:

• It generalizes to classes it never trained on. Because the loss only ever says "closer" or "farther", a model trained on some identities produces useful distances for identities it has never seen — the basis of face verification, where you cannot retrain for every new person.
• One embedding serves many tasks. Compute it once per image, then: verify by thresholding a distance, retrieve by nearest-neighbour search, cluster by feeding the vectors to any clustering method.

Two classic losses do the training, and this package implements both:

• Contrastive loss works on pairs. A same-class pair pays its squared distance (pulled together); a different-class pair pays a hinge that is zero once the pair is at least a margin apart (pushed apart, but only until far enough) — Hadsell, Chopra & LeCun (2006), "Dimensionality Reduction by Learning an Invariant Mapping", CVPR.
• Triplet loss works on triples of (anchor, positive, negative). It asks only that the negative be farther from the anchor than the positive, by at least a margin — a relative constraint, which is often easier to satisfy and to scale than pinning absolute distances — Schroff, Kalenichenko & Philbin (2015), "FaceNet: A Unified Embedding for Face Recognition and Clustering", CVPR.

A Siamese/triplet network is just one network run more than once. The two legs of a pair (or three of a triplet) share the same weights, so there is no second network to manage: stack the legs into one batch, run a single forward pass, compute the loss and its per-row gradient on the resulting embeddings, and backpropagate that gradient through the one trunk. That is exactly what Embedder.fit does — the same custom forward/loss/backward loop the rest of the family uses, with a metric loss where the classifier's softmax would be.

Install

pip install mantissa-embed

Pulls in mantissa-cnn >= 0.2.2 (and transitively mantissa-nn + the mantissa-core engine). For the demo's plots, pip install mantissa-embed[viz].

From checkouts (works today, no PyPI needed): clone this repo, cnn, mantissa-nn and mantissa side by side, build the engine (make dist there), then here:

pip install -e ../mantissa -e ../mantissa-nn -e ../cnn && pip install -e ".[viz]"

mantissa-cnn finds the sibling engine checkout automatically, and its dataset loaders find a data/ directory via MANTISSA_CNN_DATA (the demo points this at the sibling cnn/data/ for you).

Quickstart

# datasets are mantissa-cnn's; nothing downloads implicitly — fetch once:
python -m mantissa_cnn.datasets download mnist

from mantissa_cnn import datasets
from mantissa_embed import Embedder, models

Xtr, ytr, Xte, yte = datasets.subset("mnist", 6000, 2000, seed=0)

emb = Embedder(models.small_cnn_embedder(embed_dim=16), loss="triplet")  # or "contrastive"
emb.fit(Xtr, ytr, epochs=8, batch_size=64, lr=0.05, verbose=True)

Z = emb.embed(Xte)                       # (2000, 16) embedding vectors
idx, dist = emb.retrieve(Xte[0], Z, k=5) # 5 nearest test images to query 0
same = emb.verify(Xte[0], Xte[1], threshold=1.0)   # same identity? (bool)

Or compose your own trunk from mantissa-cnn's layers — any Conv/Pool/Flatten stack ending in Dense(embed_dim, act="identity"):

from mantissa_cnn import Conv2D, MaxPool2D, Flatten, Dense
from mantissa_embed import Embedder

emb = Embedder(
    [Conv2D(16, 3, pad=1), MaxPool2D(2), Flatten(), Dense(32)],  # 32-D embedding
    loss="contrastive", margin=1.0, seed=0)

The API

Embedder(layers, loss="triplet"|"contrastive", margin=None, seed=0, backend="mantissa"), then:

retrieve and verify accept raw images or a pre-computed (., embed_dim) matrix, so a fixed gallery is embedded once and reused across queries.

Deliberately minimal, like the rest of the family: NCHW float32 images, plain SGD, the two classic losses, plain Euclidean distances. No autograd graph, no optimizer zoo, no ANN index. The trunk's convolutions run in C on zero-copy float32 buffers; the losses are memory-bound reductions over embedding vectors and honestly stay in numpy (mantissa_embed.losses, importable and gradient- checked on their own). Layers allocate scratch once per batch shape and reuse it.

Results

A small_cnn_embedder (52,528 params, 16-D embedding) trained on a 6k-image MNIST subset with the triplet loss and the mantissa C engine (8 epochs, triplet loss 0.090 → 0.016), then embedding the held-out 2k test set. Reproduce with python examples/embeddings_demo.py.

The embedding separates the digits with no label ever entering the loss — colour is the true digit, shown only for the plot:

And nearest-neighbour retrieval returns same-digit images (green border = a correct match, i.e. the neighbour's true digit equals the query's):

License

MIT — © Tekin Ertekin. Base package: mantissa-cnn; engine: mantissa — same author, MIT.