hyvitasbench 1.0.0

Official implementation of HyViTas-Bench, from 'A CNN-ViT hybrid architecture search benchmark on a large-scale dataset' (IEEE Access 2025)

HyViTas-Bench

Code release for A CNN-ViT hybrid architecture search benchmark on a large-scale dataset

Getting started

Installation:

pip3 install hyvitasbench

Usage:

import torch
from hyvitasbench import *
import hyvitasbench.datasets as datasets

bench = HyViTASBench('hyvitasbench.pkl.zst')
exemplar = bench[1000]

print(exemplar.genotype)
# [[[32, 2, 3], [32, 2, 3], [32, 2, 3]], [[64, 2, 3], [64, 2, 3], [64, 2, 3]], [[112, 8, 1, 5], [112, 8, 1, 5], [112, 8, 1, 5]], [[224, 8, 1, 5], [224, 8, 1, 5], [224, 8, 1, 5]]]

# Available modes: mean, random, normal, trunc_normal, raw
print(exemplar.evaluate(mode='mean'))
# 67.164

print(exemplar.evaluate(mode='mean', dataset=datasets.OOD_RED_IMAGENET_SKETCH))
# 6.784834669601216

for dev_id, dev_name in bench.device_names.items():
    print(f"{dev_id:20s} : {dev_name}")
# aarch64_rpizero2w    : Raspberry Pi Zero 2W
# aarch64_rpi4         : Raspberry Pi 4
# tegra_orin-nano      : Jetson Orin Nano
# ...

print(exemplar.get_inference_time('tegra_orin-nano'))
# 19.04989266

net = exemplar.instantiate(device=torch.device('cuda'))  # Instantiate PyTorch model
print(net)
# Network(
# ...
# )

other_exemplar = bench.lookup([
    [[48, 4, 3], [48, 4, 3], [48, 4, 3]],
    [[64, 4, 3], [64, 4, 3], [64, 4, 3]],
    [[144, 8, 4, 5], [144, 8, 4, 5], [144, 8, 4, 5]],
    [[256, 8, 4, 5], [256, 8, 4, 5], [256, 8, 4, 5]]
])
print(other_exemplar.index)
# 1439

Download

The benchmark data archive can be downloaded from the repository release files.

The RedImageNet dataset can be downloaded from https://dx.doi.org/10.21227/egmb-rc53.

Contributors

• Luca Robbiano [e-mail]

Citation

@article{hyvitasbench,
    title        = {A CNN-ViT hybrid architecture search benchmark on a large-scale dataset},
    author       = {Robbiano, Luca and Pistilli, Francesca and Averta, Giuseppe},
    year         = {2025},
    journal      = {IEEE Access},
    volume       = {13},
    pages        = {209965-209979}
    doi          = {10.1109/ACCESS.2025.3642734}
    number       = {}
}

License

This codebase is released under the MIT License, except for portions of files that explicitly state otherwise. Please check in-file notices for any exceptions.