frbench 1.0.0

Unified, differentiable face-recognition module for AFR and FR research

FRBench

An easy-to-use, end-to-end differentiable face-recognition module that ships 45+ pretrained weights,

• covering 25 backbones, 9 loss functions, and 3 training datasets of different types,
• spanning generations from 2015 to 2024.

It is designed for anti-facial-recognition (AFR) research, where the transferability of attacks across backbones, loss functions, and training datasets is a central concern. But we believe that it benefits the face recognition field just as much.

Hand it any RGB image tensor (aligned or not); it detects and aligns the face(s), runs the chosen pretrained backbone, and returns embeddings ready for cosine similarity. Because the whole pipeline (detect → crop → align → backbone) is written in pure, differentiable PyTorch, gradients flow from the embeddings all the way back to the input pixels, enabling adversarial attacks and other gradient-based methods.

Table of Contents

• Highlights
• Motivation
• Installation
• Usage
• Model Zoo
• Acknowledgement
• License
• Citation

Highlights

• Unified & consistent. One module, one API, one weight format for 45+ models — no more stitching together incompatible repos.
• Differentiable end-to-end. Detection is non-differentiable, but cropping and 5-point alignment use grid_sample and a closed-form similarity transform, so the embedding is differentiable w.r.t. the raw input image.

Motivation

In anti-facial-recognition (AFR) research, one needs to understand how well an attack transfers across face-recognition backbones, loss functions, and training datasets of different types and generations. For instance, training an attack on ResNet50 and evaluating it on SwinV2-T (same loss, same data) fairly measures generalization to a next-generation architecture; training on IRSE50 and evaluating on IRSE100 isolates the effect of model size.

However, existing model zoos such as face.evoLVe and FaceX-Zoo do not offer this controlled variation over backbones, losses, and datasets. Researchers end up collecting and deploying models from many libraries, and still cannot be confident that the "Swin Transformer" in repo A is the same as the "SwinV1" in repo B.

FRBench addresses this with a single, unified, differentiable, easy-to-use PyTorch module backed by a rich set of consistently-trained pretrained weights. (The training code may be released later.)

Quickstart

import frbench
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Inputs must be RGB float tensors in [0, 255] — NOT torchvision ToTensor() [0, 1].
img = torch.rand(3, 480, 640, device=device) * 255.0

fr = frbench.FR("mobilevitv3-s", "arcface", "ms1m").to(device)
result = fr(img, l2_normalize=True)          # FREmbedResult(embeddings, indices, crops)
print(result.embeddings.shape)               # (1, 512) when one face is found

# Discover available models programmatically
for m in frbench.list_models():
    print(m.backbone, m.loss, m.dataset)

Install with pip install frbench (or pip install "frbench[demo]" for the notebook extras).

Installation

conda create -n ptfr python=3.12
conda activate ptfr
pip install frbench                 # core: torch, torchvision, pyyaml, tqdm
pip install "frbench[demo]"         # optional: pillow, matplotlib, seaborn

The core module needs torch, torchvision, pyyaml, and tqdm (for download progress bars). The demo notebook additionally uses pillow, matplotlib, and seaborn.

For local development, clone the repository and run pip install -e ".[demo]".

Weights are downloaded on demand into ~/.frbench (override with FRBENCH_CACHE). Prefetch assets with the CLI:

frbench-download --list                         # show all manifest keys
frbench-download --list-models                  # FR models only (no detectors)
frbench-download mobilefacenet_arcface_ms1m     # download one model
frbench-download --all                          # download everything
frbench-download --all --quiet                  # no progress / logs
frbench-download --refresh --list               # re-fetch manifest, then list

The public weights are downloaded directly from GitHub Releases; no GitHub login or token is required.

Progress bars use tqdm and are controlled by set_download_verbose(bool) (or --quiet on the download script); see Notes on verbosity.

Usage

Input contract: pass RGB float tensors in [0, 255] (not torchvision ToTensor() [0, 1]). The preprocessor warns if values look normalized.

See the demo notebook for a complete, runnable walkthrough.

forward / embed return a frbench.FREmbedResult named tuple with fields embeddings, indices, and crops. When no faces are found, embeddings and crops are empty tensors (never None).

Key options (see the docstring for the full list):

• need_crop=False — inputs are already-aligned 112×112 crops; skip detection.
• need_align=False — crop a (loosened, see loosen_crop) square box instead of 5-point aligning.
• keep_largest=False — return an embedding for every detected face, not just the largest.
• discard_invalid=True — drop images with no detected face (default: fall back to the whole image).
• tta=("flip_horizontal",) — test-time augmentations to average over (default: horizontal flip; doubles backbone cost). Pass tta=() to disable.
• l2_normalize=True — L2-normalize the returned embeddings for direct cosine comparison.
• eager_load=True (constructor) — download weights and build the backbone at init instead of first use.
• detections= — pass precomputed detections from FR.detect() to skip re-detection during iterative attacks.

# Detect once, then reuse detections in an optimization loop
dets = fr.detect([img1, img2])
for step in range(100):
    result = fr([img1, img2], detections=dets)
    loss = ...
    loss.backward()

Notes on devices

Inference runs natively on CUDA, MPS, and CPU. For backprop on Apple MPS, PyTorch needs a CPU fallback for a couple of ops (e.g. grid_sample's backward); the package sets PYTORCH_ENABLE_MPS_FALLBACK=1 on import, which is honored as long as the package is imported before torch. If you import torch first, set it yourself beforehand:

import os; os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
import torch

Notes on configuration

Settings can be overridden via environment variables or module constants in frbench/_config.py:

Setting	Env var	Default
Cache directory	FRBENCH_CACHE	~/.frbench
GitHub repo	FRBENCH_REPO	HKU-TASR/FRBench
Release tag	FRBENCH_RELEASE	weights-v1.0.0

Module-level frbench.CACHE, frbench.REPO, and frbench.RELEASE are also available.

The face detector defaults to retinaface_mobilenetv1 and can be changed per model via FR(..., detector="retinaface_resnet50").

Notes on verbosity

Output has two independent switches, both enabled by default and toggleable on the fly:

import frbench

frbench.set_warnings(False)          # silence inference warnings (e.g. no face detected)
frbench.set_download_verbose(False)  # silence download status + progress bars
frbench.set_verbose(False)           # convenience: both at once

• Inference warnings (no face detected, unsupported TTA, etc.) go through Python's standard warnings module under the frbench.FRBenchWarning category, so you can also filter or capture them with the usual tools, e.g. warnings.filterwarnings("ignore", category=frbench.FRBenchWarning) or with warnings.catch_warnings(record=True) as w: ....
• Download verbosity (log messages and the tqdm progress bar) is a separate channel controlled by set_download_verbose (or --quiet on frbench-download).
• Critical problems (invalid input tensors, missing manifest keys, failed downloads) raise exceptions (ValueError, frbench.FRBenchDownloadError, etc.) regardless of the switches above.

Notes on update checks

On the first FR(...) construction or frbench-download command in a process, FRBench checks whether a newer package or weights release is available. Results are cached for 24 hours, network failures are ignored, and no check runs merely from importing the package.

Set FRBENCH_NO_UPDATE_CHECK=1 or call frbench.set_update_check(False) to disable these reminders. Installed versions remain pinned to the weights release they were tested with; updating is always explicit.

Model Zoo

Every model is addressed by the same three manifest keys you pass to FR(backbone_type, loss_type, dataset_type). All accuracies below are in percent.

Name	backbone_type	loss_type	dataset_type	LFW	CPLFW	CALFW	CFP-FP	AgeDB-30	MegaFace (1M; Rank-1 Id)	MegaFace (1M; TAR@FAR=1e-4 Ver)	IJB-C (N1D1F1; TAR@FAR=1e-4 Ver)
ResNet100-ArcFace-MS1M	resnet-100	arcface	ms1m	99.75	92.18	95.73	97.3	96.37	93.19	98.49	95.21
ResNetV2_100-ArcFace-MS1M	resnetv2-100	arcface	ms1m	99.83	93.6	95.77	98.29	97.2	96.72	99.26	95.5
DenseNet201-ArcFace-MS1M	densenet-201	arcface	ms1m	99.85	93.42	96	98.56	97.92	97.75	99.38	95.91
IRSE100-ArcFace-MS1M	irse-100	arcface	ms1m	99.77	93.08	95.87	98.3	97.7	97.48	99.61	96.31
EfficientNet_b4-ArcFace-MS1M	efficientnetv1-b4	arcface	ms1m	99.73	91.83	95.72	97.33	96.95	95.89	99.08	94.98
MobileFaceNet_ECA-ArcFace-MS1M	mobilefacenet	arcface	ms1m	99.68	90.85	95.67	95.2	96.82	93.93	98.41	93.23
SwinV1_B-ArcFace-MS1M	swinv1-b	arcface	ms1m	99.8	93.7	96.05	98.64	98.03	98.8	99.56	96.8
ConvNeXt_B-ArcFace-MS1M	convnext-b	arcface	ms1m	99.83	93.7	96.02	98.79	97.75	98.61	99.49	96.35
ConvNeXtV2_B-ArcFace-MS1M	convnextv2-b	arcface	ms1m	99.8	93.82	96.03	98.8	97.65	98.63	99.51	96.41
SwinMLP_B-ArcFace-MS1M	swinmlp-b	arcface	ms1m	99.82	93.27	96.22	98.69	97.95	98.86	99.55	96.78
SwinV2_B-ArcFace-MS1M	swinv2-b	arcface	ms1m	99.82	93.48	96.25	98.69	97.88	98.67	99.54	96.49
MobileViTV1_S-ArcFace-MS1M	mobilevit-s	arcface	ms1m	99.5	90.47	95.43	94.71	95.67	90.83	97.74	92.82
MobileViTV2_2.0-ArcFace-MS1M	mobilevitv2-2.0	arcface	ms1m	99.5	91.03	95.58	96.24	96.23	94.62	98.69	93.97
MobileViTV3V1_S-ArcFace-MS1M	mobilevitv3-s	arcface	ms1m	99.68	90.95	95.53	95.21	96.27	92.22	98.08	93.4
MobileViTV3V2_2.0-ArcFace-MS1M	mobilevitv3-2.0	arcface	ms1m	99.75	91.8	95.7	96.87	96.67	95.79	98.97	94.48
MobileNetV1_W1-ArcFace-MS1M	mobilenet-w1	arcface	ms1m	99.58	90.7	95.8	95.96	96.97	95.02	98.64	93.6
MobileNetV2_W1-ArcFace-MS1M	mobilenetv2-w1	arcface	ms1m	99.52	89.72	95.5	93.73	96.77	92.48	97.96	92.82
MobileNetV3_L-ArcFace-MS1M	mobilenetv3-l	arcface	ms1m	99.57	90.15	95.7	94.71	96.98	93.65	98.3	92.88
MobileNetV4_Conv_M-ArcFace-MS1M	mobilenetv4conv-m	arcface	ms1m	99.72	91.63	96.03	96.74	97.38	96	99.02	94.63
SwinV2_T-ArcFace-MS1M	swinv2-t	arcface	ms1m	99.78	92.67	96.05	97.99	97.58	97.92	99.42	96
SwinV2_S-ArcFace-MS1M	swinv2-s	arcface	ms1m	99.8	93.37	95.88	98.36	97.93	98.23	99.43	96.33
SwinV2_L-ArcFace-MS1M	swinv2-l	arcface	ms1m	99.78	93.95	96.05	98.81	98.18	98.84	99.58	96.78
IRSE18-ArcFace-MS1M	irse-18	arcface	ms1m	99.47	91.57	95.88	96.66	96.87	94.47	98.57	94.04
IRSE34-ArcFace-MS1M	irse-34	arcface	ms1m	99.8	92.78	95.95	97.93	97.72	96.92	99.24	95.63
IRSE50-ArcFace-MS1M	irse-50	arcface	ms1m	99.8	93.27	95.97	98.21	97.73	97.26	99.29	96.07
IRSE100-Triplet-MS1M	irse-100	tripletloss	ms1m	99.78	91.97	95.47	97.93	97.1	91.05	98.27	91.17
IRSE100-Center-MS1M	irse-100	centerloss	ms1m	99.77	92.98	95.92	98.49	97.32	96.24	99.32	93.59
IRSE100-SphereFace-MS1M	irse-100	sphereface	ms1m	99.8	93.95	96.23	98.97	98.33	98.18	99.49	96.6
IRSE100-CosFace-MS1M	irse-100	cosface	ms1m	99.8	93.9	96.07	98.89	98.28	98.83	99.59	97
IRSE100-CurricularFace-MS1M	irse-100	curricularface	ms1m	99.82	93.82	95.98	98.8	98.15	98.93	99.59	96.98
IRSE100-MagFace-MS1M	irse-100	magface	ms1m	99.82	93.67	95.98	98.86	98.47	98.94	99.58	96.89
IRSE100-AdaFace-MS1M	irse-100	adaface	ms1m	99.82	93.72	96.17	98.76	98.23	98.88	99.61	96.96
IRSE100-UniFace-MS1M	irse-100	uniface	ms1m	99.8	93.43	96.13	98.64	98.3	98.77	99.57	96.87
SwinV2_B-Triplet-MS1M	swinv2-b	tripletloss	ms1m	99.77	93.2	95.88	98.36	97.05	94.65	98.97	93.66
SwinV2_B-Center-MS1M	swinv2-b	centerloss	ms1m	99.55	89.93	93.92	97.4	95.57	93.75	97.54	82.23
SwinV2_B-SphereFace-MS1M	swinv2-b	sphereface	ms1m	99.85	93.8	96.28	98.47	97.82	97.71	99.38	96.28
SwinV2_B-CosFace-MS1M	swinv2-b	cosface	ms1m	99.82	93.8	96.25	98.63	97.8	98.36	99.49	96.63
SwinV2_B-CurricularFace-MS1M	swinv2-b	curricularface	ms1m	99.8	93.63	96.07	98.43	97.93	98.56	99.47	96.6
SwinV2_B-MagFace-MS1M	swinv2-b	magface	ms1m	99.8	93.87	96.07	98.54	97.95	98.62	99.5	96.5
SwinV2_B-AdaFace-MS1M	swinv2-b	adaface	ms1m	99.82	93.62	96	98.27	97.72	98.66	99.5	96.64
SwinV2_B-UniFace-MS1M	swinv2-b	uniface	ms1m	99.75	93.55	95.92	98.51	97.6	98.3	99.42	96.51
IRSE100-ArcFace-WebFace4M	irse-100	arcface	webface4m	99.68	92.97	95.4	98.3	96.12	95.19	98.88	95.13
IRSE100-ArcFace-Glint360k	irse-100	arcface	glint360k	99.8	94.07	95.88	98.81	97.62	98.25	99.47	96.71
SwinV2_B-ArcFace-WebFace4M	swinv2-b	arcface	webface4m	99.82	94.42	95.75	98.69	97.58	97.14	99.39	96.82
SwinV2_B-ArcFace-Glint360k	swinv2-b	arcface	glint360k	99.85	95.03	96.07	99.03	98.3	98.81	99.63	97.43

Coverage of backbones

Under the same loss function (ArcFace) and the same dataset (MS1M), we have:

1. SOTA backbones from 2015-2024

   Backbone	Year	Venue
   ResNet	2015	NIPS
   ResNetV2	2016	ECCV
   DenseNet	2017	CVPR
   SE-Net(IRSE)	2018	CVPR
   EfficientNet	2019	ICML
   ECA-Net(MobileFaceNet_ECA)	2020	CVPR
   SwinV1	2021	ICCV
   ConvNeXtV1	2022	CVPR
   ConvNeXtV2	2023	CVPR
   MobileNetV4	2024	ECCV

2. Backbones of the same family across generations

   Backbone	Family	Paradigm
   SwinMLP	Swin Transformer	Transformer
   SwinV1	Swin Transformer	Transformer
   SwinV2	Swin Transformer	Transformer
   MobileViTV1	MobileViT	Transformer
   MobileViTV2	MobileViT	Transformer
   MobileViTV3V1	MobileViT	Transformer
   MobileViTV3V2	MobileViT	Transformer
   ConvNeXtV1	ConvNeXt	CNN
   ConvNeXtV2	ConvNeXt	CNN
   MobileNetV1	MobileNet	CNN
   MobileNetV2	MobileNet	CNN
   MobileNetV3	MobileNet	CNN
   MobileNetV4-Conv	MobileNet	CNN

3. Backbones of the same type across sizes

   Backbone	Type	Parameter Size (M)	Paradigm
   SwinV2-T	SwinV2	47.099	Transformer
   SwinV2-S	SwinV2	68.57	Transformer
   SwinV2-B	SwinV2	112.926	Transformer
   SwinV2-L	SwinV2	234.078	Transformer
   IRSE18	IRSE	24.115	CNN
   IRSE34	IRSE	34.303	CNN
   IRSE50	IRSE	43.824	CNN
   IRSE100	IRSE	65.549	CNN

Coverage of losses

Under the same backbone (IRSE100 for CNN, SwinV2-B for Transformer) and the same dataset (MS1M), we have SOTA loss functions from 2015-2023

Loss	Year	Venue
Triplet Loss	2015	CVPR
Center Loss	2016	ECCV
SphereFace	2017	CVPR
CosFace/LMCL	2018	CVPR
ArcFace	2019	CVPR
CurricularFace	2020	CVPR
MagFace	2021	CVPR
AdaFace	2022	CVPR
UniFace	2023	ICCV

Coverage of datasets

Under the same backbone (IRSE100 for CNN, SwinV2-B for Transformer) and the same loss function (ArcFace), we have 3 datasets of varying image and identity counts:

Dataset	Images	Identities
MS1M	5,822,653	85,742
WebFace4M	4,235,242	205,990
Glint360k	17,091,657	360,232

Acknowledgement

Codes in frbench/backbones/ and frbench/utils/retinaface.py are adapted from existing open-source projects:

• timm
• cavaface
• FaceX-Zoo
• InsightFace
• facebookresearch/ConvNeXt
• facebookresearch/ConvNeXt-V2
• apple/ml-cvnets
• micronDLA/MobileViTv3
• jaiwei98/mobile-vit-pytorch
• microsoft/Swin-Transformer
• HKU-TASR/Protego

License

This project is licensed under the MIT License.

Citation

If you find our project useful in your research, please consider citing:

@inproceedings{wang2026protego,
    title={Protego: User-Centric Pose-Invariant Privacy Protection Against Face Recognition-Induced Digital Footprint Exposure},
    author={Ziling Wang and Shuya Yang and Jialin Lu and Ka-Ho Chow},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year={2026}
}