Skip to main content

A collection of DNN test input prioritizers,in particular neuron coverage and surprise adequacy.

Project description

DNN-TIP: Common Test Input Prioritizers Library

test Code style: black docstr-coverage Imports: isort Python Version PyPi Deployment License DOI

Implemented Approaches

  • Surprise Adequacies
    • Distance-based Surprise Adequacy (DSA)
    • Likelihood-based Surprise Adequacy (LSA)
    • MultiModal-Likelihood-based Surprise Adequacy (MLSA)
    • Mahalanobis-based Surprise Adequacy (MDSA)
    • abstract MultiModal Surprise Adequacy
  • Surprise Coverage
    • Neuron-Activation Coverage (NAC)
    • K-Multisection Neuron Coverage (KMNC)
    • Neuron Boundary Coverage (NBC)
    • Strong Neuron Activation Coverage (SNAC)
    • Top-k Neuron Coverage (TKNC)
  • Utilities
    • APFD calculation
    • Coverage-Added and Coverage-Total Prioritization Methods (CAM and CTM)

If you are looking for the uncertainty metrics we also tested (including DeepGini), head over to the sister repository uncertainty-wizard.

If you want to reproduce our exact experiments, there's a reproduction package and docker stuff available at testingautomated-usi/simple-tip.

Installation

It's as easy as pip install dnn-tip.

Documentation

Find the documentation at https://testingautomated-usi.github.io/dnn-tip/.

Citation

Here's the reference to the paper as part of which this library was release:

@inproceedings{10.1145/3533767.3534375,
author = {Weiss, Michael and Tonella, Paolo},
title = {Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)},
year = {2022},
isbn = {9781450393799},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3533767.3534375},
doi = {10.1145/3533767.3534375},
booktitle = {Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis},
pages = {139–150},
numpages = {12},
keywords = {neural networks, Test prioritization, uncertainty quantification},
location = {Virtual, South Korea},
series = {ISSTA 2022}
}


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dnn-tip-0.1.1.tar.gz (15.5 kB view details)

Uploaded Source

Built Distribution

dnn_tip-0.1.1-py3-none-any.whl (15.6 kB view details)

Uploaded Python 3

File details

Details for the file dnn-tip-0.1.1.tar.gz.

File metadata

  • Download URL: dnn-tip-0.1.1.tar.gz
  • Upload date:
  • Size: 15.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.8

File hashes

Hashes for dnn-tip-0.1.1.tar.gz
Algorithm Hash digest
SHA256 c89eb227329b3b264b3eccbb1812ca12058245862a28948ff928edb683505281
MD5 84dfdc4fdbdbe88e243d93689d7f4398
BLAKE2b-256 d4f3714e5017632ca779fd548a89b54e181e8d2a5457270954b4b6a3398bbc08

See more details on using hashes here.

File details

Details for the file dnn_tip-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: dnn_tip-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 15.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.8

File hashes

Hashes for dnn_tip-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 544763e798e6db1289296f673214bbb57e6878c0f9489041df10cc8771df917a
MD5 ebf97405bd2fc8ea205aeb1d5fbd9130
BLAKE2b-256 af9e56e68c88b4059581179cdd0d56f2652dc381374ee754e9c9a2303698b668

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page