Skip to main content

Model evaluation without manual labels

Project description

Logo
MOVAL

Estimating performance for safe deployment of machine learning models

MOVAL is a Python package designed for assessing model performance in the absence of ground truth labels. It computes and calibrated confidence scores to accurately reflect the likelihood of predictions, leveraging these calibrated confidence scores to estimate the model's overall performance. Notably, MOVAL operates without the need for ground truth labels in the target domains and supports the evaluation of model performance in classification, 2D segmentation, and 3D segmentation.

MOVAL highlights a key feature—class-wise calibration, recognized as essential for addressing long-tailed distributions commonly found in real-world datasets. This proves especially significant in segmentation tasks where background samples often outnumber foregrounds. The inclusion of class-specific variants becomes crucial for accurately estimating segmentation performance. Additionally, MOVAL offers support for various types of confidence scores, enhancing its versatility.

What it offers:

User Document

The latest documentation can be found here.

Reference

@inproceedings{li2022estimating,
  title={Estimating model performance under domain shifts with class-specific confidence scores},
  author={Li, Zeju and Kamnitsas, Konstantinos and Islam, Mobarakol and Chen, Chen and Glocker, Ben},
  booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
  pages={693--703},
  year={2022},
  organization={Springer}
}

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

moval-0.3.25.tar.gz (37.7 kB view details)

Uploaded Source

Built Distribution

moval-0.3.25-py3-none-any.whl (35.1 kB view details)

Uploaded Python 3

File details

Details for the file moval-0.3.25.tar.gz.

File metadata

  • Download URL: moval-0.3.25.tar.gz
  • Upload date:
  • Size: 37.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.0

File hashes

Hashes for moval-0.3.25.tar.gz
Algorithm Hash digest
SHA256 1d2b80df5135201c101d8d5af7a27e2f86da50851c29585635a576466e4e0e55
MD5 e46b875d4b46bfb71c8868c9ec1c3fc5
BLAKE2b-256 d9c6669e5f310e930327eec9550a864aa6243790368fc4d54cfe035ff27f4a12

See more details on using hashes here.

Provenance

File details

Details for the file moval-0.3.25-py3-none-any.whl.

File metadata

  • Download URL: moval-0.3.25-py3-none-any.whl
  • Upload date:
  • Size: 35.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.8.0

File hashes

Hashes for moval-0.3.25-py3-none-any.whl
Algorithm Hash digest
SHA256 cb6b81f7dc8bbcd5594af5bf9ab07c1feb6284e8c17ba8a935c320ef714048d9
MD5 2bde8f9fd462acb0634deb351fc06f43
BLAKE2b-256 63b4992c6fa8ff3637c6f66b090a8236539d80907dc4e1fc44093886c5a3585b

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page