ua-metrics

ua-metrics is a Python package for standard and uncertainty-adjusted regression metrics, accompanying the paper "Uncertainty-Aware Metrics for Evaluating Machine Learning Regression Models in Materials Testing".

The package provides three uncertainty-model modules:

• ua_metrics.gaussian
• ua_metrics.student_t
• ua_metrics.lognormal

Features

• Standard regression metrics.
• Uncertainty-adjusted regression metrics.
• Gaussian, Student-t, and lognormal uncertainty models.
• Heteroscedastic and homoscedastic uncertainty handling.
• A consistent module-based API.

Installation

pip install ua-metrics

Quick start

from ua_metrics import gaussian as gau
from ua_metrics import lognormal as logn
from ua_metrics import student_t as t

Default behavior (heteroscedastic uncertainty, mean scaling) and its equivalent explicit form:

v1 = gau.mae_ua([100, 110, 95], [102, 108, 97], 5.0)

v2 = gau.rmse_ua(
    [100, 110, 95],
    [102, 108, 97],
    5.0,
    mode="hetero",
    scale="mean",
)

Homoscedastic uncertainty with a constant absolute standard deviation:

v3 = gau.mae_ua([100, 110, 95], [102, 108, 97], 1.0, mode="homo")

Student-t and lognormal uncertainty models:

v4 = t.mae_ua([100, 110, 95], [102, 108, 97], 7.5, df=3.0)
v5 = logn.mae_ua([100, 110, 95], [102, 108, 97], 12.0)

Package structure

Standard metrics

from ua_metrics import mae, median_absolute_error, mse, rmse
from ua_metrics import mape, smape, r2_score, adjusted_r2_score

Uncertainty-adjusted metrics

from ua_metrics import gaussian as gau
from ua_metrics import student_t as t
from ua_metrics import lognormal as logn

Each uncertainty module provides:

• mae_ua
• median_absolute_error_ua
• mse_ua
• rmse_ua
• mape_ua
• smape_ua
• r2_score_ua
• adjusted_r2_score_ua

Interface

All uncertainty-adjusted metrics use the same public interface:

metric_ua(y_obs, y_pred, value, *, mode="hetero", scale="mean", ...)

Argument semantics

• value with mode="hetero" is interpreted as CV percent of uncertainty.
• value with mode="homo" is interpreted as a constant absolute uncertainty standard deviation.

Default behavior

The default configuration is:

• mode="hetero"
• scale="mean"

Under this default, value=5.0 means the uncertainty standard deviation is 5% of the observation-wise mean scale.

Scale definitions

Supported scale values are:

• "mean", defined as 0.5 * (y_obs + y_pred)
• "y_obs"
• "y_pred"

Notes

• Values greater than 100 are allowed in heteroscedastic mode.
• Homoscedastic mode expects a scalar constant absolute uncertainty value.

Citation

If you use ua-metrics in your work, please cite the accompanying paper (accepted for publication in Measurement, in press — volume/pages/DOI to be added once assigned) and/or the software itself. Machine-readable citation metadata is kept up to date in CITATION.cff.

@article{abbas_ua_metrics_paper,
  title   = {Uncertainty-Aware Metrics for Evaluating Machine Learning Regression Models in Materials Testing},
  author  = {Abbas, Mohammad and Zaumanis, Martins},
  journal = {Measurement},
  year    = {2026},
  note    = {Accepted, in press. Full citation to be updated once volume/pages/DOI are assigned.}
}

@software{abbas_ua_metrics_software,
  title   = {ua-metrics},
  author  = {Abbas, Mohammad},
  year    = {2026},
  version = {1.0.0},
  url     = {https://github.com/Moha-Abbas/ua-metrics}
}

License

MIT — see LICENSE.