pytorch-metric-learning 0.9.66

The easiest way to use metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.

PyTorch Metric Learning

Documentation

View the documentation here

Benefits of this library

1. Ease of use
   • Add metric learning to your application with just 2 lines of code in your training loop.
   • Mine pairs and triplets with a single function call.
2. Flexibility
   • Mix and match losses, miners, and trainers in ways that other libraries don't allow.

Installation:

Conda:

conda install pytorch-metric-learning -c metric-learning

Pip:

pip install pytorch-metric-learning

Benchmark results

See powerful-benchmarker to view benchmark results and to use the benchmarking tool.

Currently implemented classes:

Loss functions:

• AngularLoss (Deep Metric Learning with Angular Loss)
• ArcFaceLoss (ArcFace: Additive Angular Margin Loss for Deep Face Recognition)
• ContrastiveLoss
• FastAPLoss (Deep Metric Learning to Rank)
• GeneralizedLiftedStructureLoss
• MarginLoss (Sampling Matters in Deep Embedding Learning)
• MultiSimilarityLoss (Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning)
• NCALoss (Neighbourhood Components Analysis)
• NormalizedSoftmaxLoss (Classification is a Strong Baseline for DeepMetric Learning)
• NPairsLoss (Improved Deep Metric Learning with Multi-class N-pair Loss Objective)
• ProxyNCALoss (No Fuss Distance Metric Learning using Proxies)
• SignalToNoiseRatioContrastiveLoss (Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning)
• SoftTripleLoss (SoftTriple Loss: Deep Metric Learning Without Triplet Sampling)
• TripletMarginLoss
• more to be added

Mining functions:

• AngularMiner
• BatchHardMiner (In Defense of the Triplet Loss for Person Re-Identification)
• DistanceWeightedMiner (Sampling Matters in Deep Embedding Learning)
• HDCMiner (Hard-Aware Deeply Cascaded Embedding)
• MaximumLossMiner
• MultiSimilarityMiner (Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning)
• PairMarginMiner
• TripletMarginMiner
• more to be added

Regularizers:

• RegularFaceRegularizer (RegularFace: Deep Face Recognition via Exclusive Regularization)

Samplers:

• MPerClassSampler
• FixedSetOfTriplets
• more to be added

Training methods:

• MetricLossOnly
• TrainWithClassifier
• CascadedEmbeddings (Hard-Aware Deeply Cascaded Embedding)
• DeepAdversarialMetricLearning (Deep Adversarial Metric Learning)
• UnsupervisedEmbeddingsUsingAugmentations
• more to be added

Testing methods:

• GlobalEmbeddingSpaceTester
• WithSameParentLabelTester
• more to be added

Overview

Let’s try the vanilla triplet margin loss. In all examples, embeddings is assumed to be of size (N, embedding_size), and labels is of size (N).

from pytorch_metric_learning import losses
loss_func = losses.TripletMarginLoss(margin=0.1)
loss = loss_func(embeddings, labels)

Loss functions typically come with a variety of parameters. For example, with the TripletMarginLoss, you can control how many triplets per sample to use in each batch. You can also use all possible triplets within each batch:

loss_func = losses.TripletMarginLoss(triplets_per_anchor="all")

Sometimes it can help to add a mining function:

from pytorch_metric_learning import miners, losses
miner = miners.MultiSimilarityMiner(epsilon=0.1)
loss_func = losses.TripletMarginLoss(margin=0.1)
hard_pairs = miner(embeddings, labels)
loss = loss_func(embeddings, labels, hard_pairs)

In the above code, the miner finds positive and negative pairs that it thinks are particularly difficult. Note that even though the TripletMarginLoss operates on triplets, it’s still possible to pass in pairs. This is because the library automatically converts pairs to triplets and triplets to pairs, when necessary.

In general, all loss functions take in embeddings and labels, with an optional indices_tuple argument (i.e. the output of a miner):

# From BaseMetricLossFunction
def forward(self, embeddings, labels, indices_tuple=None)

And (almost) all mining functions take in embeddings and labels:

# From BaseMiner
def forward(self, embeddings, labels)

For more complex approaches, like deep adversarial metric learning, use one of the trainers.

To check the accuracy of your model, use one of the testers. Which tester should you use? Almost definitely GlobalEmbeddingSpaceTester, because it does what most metric-learning papers do.

Also check out the example scripts. Each one shows how to set up models, optimizers, losses etc for a particular trainer.

To learn more about all of the above, see the documentation.

Acknowledgements

Thank you to Ser-Nam Lim at Facebook AI, and my research advisor, Professor Serge Belongie. This project began during my internship at Facebook AI where I received valuable feedback from Ser-Nam, and his team of computer vision and machine learning engineers and research scientists.

Citing this library

If you'd like to cite pytorch-metric-learning in your paper, you can use this bibtex:

@misc{Musgrave2019,
  author = {Musgrave, Kevin and Lim, Ser-Nam and Belongie, Serge},
  title = {PyTorch Metric Learning},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/KevinMusgrave/pytorch-metric-learning}},
}