ood-detection 0.0.3

A tutorial of NBDev, its use cases and making libraries from it.

OOD Detection

This library is used for OOD Detection where a model encounters new classes at test time that were not seen during training. The goal is to detect that such inputs do not belong to any of the training classes.

flowchart LR
  A[Feature\nEmbeddings] --> C{OOD Detection}
  B[In Distribution\nLabels] --> C
  C --> F[Uncertainty Score]
  F --> D[Out of Distribtuion]
  F --> E[In Distribtuion]

Install

pip install ood_detection

Example Usage

import numpy as np
from ood_detection.core import OODMetric # imports OODMetric class and other utility functions

train_embedding = np.random.standard_normal((32, 2048))
train_labels = np.random.randint(low=0, high=5, size=(32,))

ood = OODMetric(train_embedding, train_labels)

test_embedding = np.random.standard_normal((32, 2048)) # test embedding from same distribution
scores = ood.compute_rmd(test_embedding) # compute relative mahalanobis distance
print(scores)

[ 1.16065497e+13 -1.37269901e+13  3.54920865e+12  4.75570475e+12
 -4.90615930e+12 -2.63622848e+12 -5.22489520e+11 -7.67105637e+12
  1.30991140e+12 -5.38689280e+12  2.71026479e+12 -4.07842659e+13
 -1.01482832e+13 -2.18136787e+13 -6.53841964e+12 -1.70525347e+13
  1.06493867e+13 -2.04729993e+13  4.68809372e+12 -6.11747086e+12
  1.09862330e+13  1.03001857e+13 -2.91312276e+13 -9.26086735e+12
  7.23079505e+12  7.26673743e+12 -4.73734980e+13  3.17798849e+12
  1.99687662e+13  2.99860166e+12  9.86244208e+11  8.76676896e+12]

Built using NBDev

This OOD Detection library was built in a jupyter notebook with proper documentation and test cases. These test cases are verified before they are published to Github Pages, PyPi, Conda etc.

I’ve written down a NBDev Tutorial explaining the thought process of Jeremy Howard and folks at FastAI behind building it. The tutorial covers about what it can do and how it is an amazing tool with support for building softwares following the best coding principles.