Eyeglasses and sunglasses detector (classifier and segmenter)
Project description
Glasses Detector
Eyeglasses and sunglasses classifier + glasses and their frames segmenter. This project provides a quick way to use the pre-trained models via python script or terminal. Based on selected task, an image or a directory of images will be processed and corresponding labels or masks will be generated.
Note: the project is BETA stage. Currently, only sunglasses classification models (except huge) and full glasses segmentation models (except medium) are available.
Installation
Minimum version of Python 3.10 is required. Also, you may want to install Pytorch in advance for your device to enable GPU support. Note that CUDA is backwards compatible, thus even if you have the newest version of CUDA Toolkit, Pytorch should work just fine.
To install the package, simply run:
pip install glasses-detector
You can also install it from source:
git clone https://github.com/mantasu/glasses-detector
cd glasses-detector && pip install .
If you want to train your own models on the same datasets (or your custom ones), check the GitHub repository.
Features
There are 2 kinds of classifiers and 2 kinds of segmenters (terminology is a bit off but easier to handle with unique names):
- Eyeglasses classifier - identifies only transparent glasses, i.e., prescription spectacles.
- Sunglasses classifier - identifies only occluded glasses, i.e., sunglasses.
- Full glasses segmenter - segments full glasses, i.e., their frames and actual glasses (regardless of the glasses type).
- Glasses frames segmenter - segments glasses frames (regardless of the glasses type).
Each kind has 5 different model architectures with naming conventions set from tiny to huge.
Classification
A classifier only identifies whether a corresponding category of glasses (transparent eyeglasses or occluded sunglasses) is present:
Model type | |||
---|---|---|---|
Eyeglasses classifier | wears | doesn't wear | doesn't wear |
Sunglasses classifier | doesn't wear | wears | doesn't wear |
Any glasses classifier | wears | wears | doesn't wear |
These are the performances of eyeglasses and sunglasses models and their sizes. Note that the joint glasses classifier would have an average accuracy and a combined model size of both eyeglasses and sunglasses models.
Eyeglasses classification models (performance & weights)
Model type | BCE loss $\downarrow$ | F1 score $\uparrow$ | ROC-AUC score $\uparrow$ | Num params $\downarrow$ | Model size $\downarrow$ |
---|---|---|---|---|---|
Eyeglasses classifier tiny | TBA | TBA | TBA | TBA | TBA |
Eyeglasses classifier small | TBA | TBA | TBA | TBA | TBA |
Eyeglasses classifier medium | TBA | TBA | TBA | TBA | TBA |
Eyeglasses classifier large | TBA | TBA | TBA | TBA | TBA |
Eyeglasses classifier huge | TBA | TBA | TBA | TBA | TBA |
Sunglasses classification models (performance & weights)
Model type | BCE loss $\downarrow$ | F1 score $\uparrow$ | ROC-AUC score $\uparrow$ | Num params $\downarrow$ | Model size $\downarrow$ |
---|---|---|---|---|---|
Sunglasses classifier tiny | 0.1149 | 0.9137 | 0.9967 | 27.53 k | 0.11 Mb |
Sunglasses classifier small | 0.0645 | 0.9434 | 0.9987 | 342.82 k | 1.34 Mb |
Sunglasses classifier medium | 0.0491 | 0.9651 | 0.9992 | 1.52 M | 5.84 Mb |
Sunglasses classifier large | 0.0532 | 0.9685 | 0.9990 | 4.0 M | 15.45 Mb |
Sunglasses classifier huge | TBA | TBA | TBA | TBA | TBA |
Segmentation
A full-glasses segmenter generates masks of people wearing corresponding categories of glasses and their frames, whereas frames-only segmenter generates corresponding masks but only for glasses frames:
Model type | |||
---|---|---|---|
Full/frames eyeglasses segmenter | |||
Full/frames sunglasses segmenter | |||
Full/frames any glasses segmenter |
There is only one model group for each full-glasses and frames-only segmentation tasks. Each group is trained for both eyeglasses and sunglasses. Although you can use it as is, it is only one part of the final full-glasses or frames-only segmentation model - the other part is a specific classifier, therefore, the accuracy and the model size would be a combination of the generic (base) segmenter and a classifier of a specific glasses category.
Full glasses segmentation models (performance & weights)
Model type | BCE loss $\downarrow$ | F1 score $\uparrow$ | Dice score $\uparrow$ | Num params $\downarrow$ | Model size $\downarrow$ |
---|---|---|---|---|---|
Full glasses segmenter tiny | 0.0580 | 0.9054 | 0.9220 | 926.07 k | 3.54 Mb |
Full glasses segmenter small | 0.0603 | 0.8990 | 0.9131 | 3.22 M | 12.37 Mb |
Full glasses segmenter medium | TBA | TBA | TBA | TBA | TBA |
Full glasses segmenter large | 0.0515 | 0.9152 | 0.9279 | 32.95 M | 125.89 Mb |
Full glasses segmenter huge | 0.0516 | 0.9147 | 0.9272 | 58.63 M | 224.06 Mb |
Glasses frames segmentation models (performance & weights)
Model type | BCE loss $\downarrow$ | F1 score $\uparrow$ | Dice score $\uparrow$ | Num params $\downarrow$ | Model size $\downarrow$ |
---|---|---|---|---|---|
Glasses frames segmenter tiny | TBA | TBA | TBA | TBA | TBA |
Glasses frames segmenter small | TBA | TBA | TBA | TBA | TBA |
Glasses frames segmenter medium | TBA | TBA | TBA | TBA | TBA |
Glasses frames segmenter large | TBA | TBA | TBA | TBA | TBA |
Glasses frames segmenter huge | TBA | TBA | TBA | TBA | TBA |
Examples
Command Line
You can run predictions via the command line. For example, classification of a single or multiple images, can be performed via
glasses-detector -i path/to/img --kind sunglasses-classifier # Prints 1 or 0
glasses-detector -i path/to/dir --kind sunglasses-classifier # Generates CSV
Running segmentation is similar, just change the kind
argument:
glasses-detector -i path/to/img -k glasses-segmenter # Generates img_mask file
glasses-detector -i path/to/dir -k glasses-segmenter # Generates dir with masks
Note: you can also specify things like
--output-path
,--label-type
,--size
,--device
etc. Use--glasses-detector -h
for more details or check the documentation page.
Python Script
You can import the package and its models via the python script for more flexibility. Here is an example of how to classify people wearing sunglasses (will generate an output file where each line will contain the name of the image and the predicted label, e.g., some_image.jpg,1
):
from glasses_detector import SunglassesClassifier
classifier = SunglassesClassifier(model_type="small", pretrained=True).eval()
classifier.predict(
input_path="path/to/dir",
output_path="path/to/output.csv",
label_type="int",
)
Using a segmenter is similar, here is an example of using a sunglasses segmentation model:
from glasses_detector import FullSunglassesSegmenter
# model_type can also be a tuple: (classifier size, base glasses segmenter size)
segmenter = FullSunglassesSegmenter(model_type="small", pretrained=True).eval()
segmenter.predict(
input_path="path/to/dir",
output_path="path/to/dir_masks",
mask_type="img",
)
Note: there is much more flexibility that you can do with the given models, for instance, you can use only base segmenters without accompanying classifiers, or you can define your own prediction methods without resizing images to
256x256
(as what is done in the background). For more details refer to the documentation page, for instance at how segmenter prediction method works.
Demo
Feel free to play around with some demo image files. For example, after installing through pip, you can run:
git clone https://github.com/mantasu/glasses-detector && cd glasses-detector/data
glasses-detector -i demo -o demo_labels.csv --kind sunglasses-classifier --label str
References
The following model architectures were used from Torchvision library:
- Classifier small - ShuffleNet V2 (x0.5) based on ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design paper
- Classifier medium - MobileNet V3 (small) based on Searching for MobileNetV3 paper
- Classifier large - EfficientNet B0 based on EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks paper
- Segmenter small - LRASPP based on Searching for MobileNetV3 paper
- Segmenter medium - FCN (ResNet-50) based on Fully Convolutional Networks for Semantic Segmentation paper
- Segmenter large - DeepLab V3 (ResNet-101) based on Rethinking Atrous Convolution for Semantic Image Segmentation paper
Tiny classifiers and tiny segmenters are the custom models created by me with the aim to have as few parameters as possible while still maintaining a reasonable accuracy.
Citation
@misc{glasses-detector,
author = {Mantas Birškus},
title = {Glasses Detector},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/mantasu/glasses-detector}},
doi = {TBA}
}
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for glasses_detector-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6b3e99f786eb87b16c6dafa651b7c721a0f6b37c26c523866d5bfcea1cc19ead |
|
MD5 | b217dab9c5f07c8240cd509e3757f6c2 |
|
BLAKE2b-256 | 971d1f30e13fa144084cb15da7bee80414a2cc30a561fe3dc67d2e275003069f |