aule 0.0.4

Validation metrics and plots for machine learning models, with a focus on earth observation and climate science.

aule

aule provides validation metrics and plots for machine learning model outputs, with a focus on earth observation and climate science use cases (gridded fields, ensembles, multi-temporal data).

Named after Aulë, the Vala of craft in Tolkien's mythology — the one who forges and evaluates the work of his own hands.

Supported input shapes

Every function accepts numpy arrays in one of four shapes:

• (batch, H, W, C)
• (batch, H, W, C, T)
• (H, W, C)
• (H, W, C, T)

When an array is 4D, pass data_format="bhwc" (default) or data_format="hwct" to disambiguate, since the two shapes cannot be told apart from shape alone.

Installation

pip install aule

# with cartopy support for geographic basemaps
pip install aule[geo]

Quick example

import numpy as np
from aule.metrics import rmse, mae, pearson_r, ssim
from aule.plots import plot_field_comparison, plot_scatter

gt   = np.random.rand(64, 64, 1)
pred = gt + np.random.normal(0, 0.1, gt.shape)

print(rmse(gt, pred))
print(pearson_r(gt, pred))

fig, axes = plot_field_comparison(gt, pred)
fig, ax = plot_scatter(gt, pred, save_path="scatter.png")

What's included

Metrics are organized by family in aule.metrics, all importable directly from aule.metrics:

• core: rmse, mse, mae, bias, pearson_r, ssim, psnr, r2_score, mape, smape, nse, kge, max_error, explained_variance, wasserstein_distance, quantile_mapping_bias
• spectral: spectral_error, gradient_error, psd_radial_error, spectral_angle_mapper
• climate: seasonal_error, percentile_error, pixelwise_temporal_correlation, trend_error, extreme_event_duration_error, autocorrelation_error, wet_day_frequency_error, dry_spell_error, anomaly_correlation_coefficient
• ensemble: ensemble_spread, crps, rank_histogram, brier_score, spread_skill_ratio, crps_skill_score
• earth_observation: normalized_difference_index, index_error, change_detection_error
• classification: iou, dice, precision_recall_f1, confusion_matrix_metrics, cohen_kappa (binary or multi-class, via average/num_classes)
• uncertainty: picp, pit_histogram
• spatial_verification: fractions_skill_score (neighborhood-based, displacement-tolerant), energy_score (multivariate CRPS generalization)

Plots are organized similarly in aule.plots:

• core: plot_scatter, plot_qq, plot_histogram_comparison, plot_error_histogram
• spatial: plot_field_comparison, plot_bias_map, plot_correlation_map (optional cartopy basemap via lat/lon)
• climate: plot_temporal_trend, plot_temporal_scatter
• ensemble: plot_ensemble_spread_map, plot_rank_histogram
• diagnostics: plot_taylor_diagram, plot_boxplot_comparison, plot_violin_comparison, plot_time_series, plot_error_map
• classification: plot_confusion_matrix, plot_reliability_diagram
• advanced: plot_hovmoller, plot_cdf_comparison, plot_spectral_density, plot_time_evolution

Divergent colormap normalizations

plot_bias_map, plot_error_map (signed branch), and plot_field_comparison (difference panel) expose a norm_type parameter to better highlight extreme values on divergent maps, beyond the default "linear" scaling:

• "power": power-law compression of the near-zero range (tune via gamma in norm_kwargs, default 0.5); a smooth, mild way to make extremes stand out more without disturbing background noise.
• "symlog": linear near zero (within linthresh), logarithmic beyond it; the strongest extreme-value contrast, at the cost of making near-threshold noise visible as speckle.
• "twoslope": anchors a (possibly off-zero) vcenter to the colormap midpoint, for data that isn't symmetric around zero.

from aule.plots import plot_bias_map

fig, ax = plot_bias_map(gt, pred, norm_type="symlog", norm_kwargs={"linthresh": 0.02})

The underlying normalization builders (power_norm, symlog_norm, asymmetric_twoslope_norm, and the resolve_diverging_norm dispatcher) are available directly from aule.plots._style for custom plots.

Automatic validation report

aule.report.generate_report runs a curated set of metrics and plots and assembles them into a single self-contained HTML file (no external assets, figures embedded as base64 PNGs):

from aule.report import generate_report

generate_report(gt, pred, save_path="report.html")

Object-oriented usage

Every metric and plot is also available as a method on the aule class, which binds y_true/y_pred (and optionally data_format/ignore_nan) once. New functions added to aule.metrics or aule.plots are picked up automatically, no extra wiring needed.

from aule import aule

v = aule(gt, pred)
print(v.rmse())
print(v.pearson_r())
fig, ax = v.plot_scatter(save_path="scatter.png")

Notebooks

The notebooks/ folder contains worked examples for every metric and plot family, each runnable end-to-end:

1. 01_core_metrics.ipynb
2. 02_spectral_and_earth_observation_metrics.ipynb
3. 03_climate_metrics.ipynb
4. 04_ensemble_and_uncertainty_metrics.ipynb
5. 05_classification_metrics.ipynb
6. 06_plots.ipynb
7. 07_aule_class.ipynb
8. 08_spatial_verification_and_advanced_plots.ipynb
9. 09_validation_report.ipynb
10. 10_color_normalization.ipynb

Documentation

The documentation is produced using pdoc.

python build_doc.py