A collection of DNN test input prioritizers,in particular neuron coverage and surprise adequacy.
Project description
DNN-TIP: Common Test Input Prioritizers Library
Implemented Approaches
- Surprise Adequacies
- Distance-based Surprise Adequacy (DSA)
- Likelihood-based Surprise Adequacy (LSA)
- MultiModal-Likelihood-based Surprise Adequacy (MLSA)
- Mahalanobis-based Surprise Adequacy (MDSA)
- abstract MultiModal Surprise Adequacy
- Surprise Coverage
- Neuron-Activation Coverage (NAC)
- K-Multisection Neuron Coverage (KMNC)
- Neuron Boundary Coverage (NBC)
- Strong Neuron Activation Coverage (SNAC)
- Top-k Neuron Coverage (TKNC)
- Utilities
- APFD calculation
- Coverage-Added and Coverage-Total Prioritization Methods (CAM and CTM)
If you are looking for the uncertainty metrics we also tested (including DeepGini), head over to the sister repository uncertainty-wizard.
If you want to reproduce our exact experiments, there's a reproduction package and docker stuff available at testingautomated-usi/simple-tip.
Installation
It's as easy as pip install dnn-tip
.
Documentation
Find the documentation at https://testingautomated-usi.github.io/dnn-tip/.
Citation
Here's the reference to the paper as part of which this library was release:
@inproceedings{10.1145/3533767.3534375,
author = {Weiss, Michael and Tonella, Paolo},
title = {Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)},
year = {2022},
isbn = {9781450393799},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3533767.3534375},
doi = {10.1145/3533767.3534375},
booktitle = {Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis},
pages = {139–150},
numpages = {12},
keywords = {neural networks, Test prioritization, uncertainty quantification},
location = {Virtual, South Korea},
series = {ISSTA 2022}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file dnn-tip-0.1.1.tar.gz
.
File metadata
- Download URL: dnn-tip-0.1.1.tar.gz
- Upload date:
- Size: 15.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c89eb227329b3b264b3eccbb1812ca12058245862a28948ff928edb683505281 |
|
MD5 | 84dfdc4fdbdbe88e243d93689d7f4398 |
|
BLAKE2b-256 | d4f3714e5017632ca779fd548a89b54e181e8d2a5457270954b4b6a3398bbc08 |
File details
Details for the file dnn_tip-0.1.1-py3-none-any.whl
.
File metadata
- Download URL: dnn_tip-0.1.1-py3-none-any.whl
- Upload date:
- Size: 15.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.8
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 544763e798e6db1289296f673214bbb57e6878c0f9489041df10cc8771df917a |
|
MD5 | ebf97405bd2fc8ea205aeb1d5fbd9130 |
|
BLAKE2b-256 | af9e56e68c88b4059581179cdd0d56f2652dc381374ee754e9c9a2303698b668 |