mmcr 0.1.0

MMCR Loss: Learning efficient coding of natural images with maximum manifold capacity representations

Pytorch implementation of Maximum Manifold Capacity Representations Loss

This is not an official implementation from the authors. Official implementation from the authors.

Maximum Manifold Capacity Representation Loss (MMCR Loss) is a novel objective function for self-supervised learning (SSL) proposed by researchers in Center for Neural Science, NYU.

This repository aims to offer a convenient MMCR loss module for PyTorch, which can be easily integrated into your projects using git clone or pip install.

How to install

pip3 install mmcr

or

git clone https://github.com/skyil7/mmcr
cd mmcr
pip install -e .

Usage

import torch
from mmcr import MMCRLoss

loss = MMCRLoss()

input_tensor = torch.randn((8, 16, 128))  # batch_size, n_aug, feature_dim
loss_val = loss(input_tensor)

print(loss_val)

How it works

\mathcal{L} = \lambda\frac{\sum^{N}_{i=1}\lVert z_{i} \rVert_{*}}{N} - \lVert C\rVert_{*}

Where $\lambda$ is a trade-off parameter, $\lVert z_i\rVert_*$ is local nuclear norm of the $i$-th sample's augmented matrix, and $\lVert C\rVert_*$ is the global nuclear norm of centroid matrix $C$.

Arguments

• lmbda: Trade-off parameter $\lambda$. default is 0.
• n_aug: number of augmented views. If your input tensor is 3-dimensional $(N, k, d)$, you don't need to specify it.

Original Implementation from the author

• This repository was developed with reference to the official implementation provided by the authors.