aiice-bench 0.12.0

A standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration

Aiice

---

AIICE is an open-source Python framework designed as a standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration. It provides reproducible pipelines for loading, preprocessing, and evaluating satellite-derived OSI-SAF data, supporting both short- and long-term prediction horizons

Installation

The simplest way to install framework with pip:

pip install aiice-bench

Quickstart

The AIICE class provides a simple interface for loading Arctic ice data, preparing datasets, and benchmarking PyTorch models:

from aiice import AIICE

# Initialize AIICE with a sliding window 
# of past 30 days and forecast of 7 days
aiice = AIICE(
    pre_history_len=30,
    forecast_len=7,
    batch_size=32,
    start="2022-01-01",
    end="2022-12-31"
)

# Define your PyTorch model
model = MyModel()

# Run benchmarking to compute metrics on the dataset
report = aiice.bench(model)
print(report)

Check package doc and see more usage examples. You can also explore the raw dataset and work with it independently via Hugging Face

Leaderboard

The leaderboard reports the mean performance of each model across the evaluation dataset. You can check models' setup in examples.

		baseline_mean	baseline_repeat
Barents Sea	bin_accuracy	0.874963	0.848936
iou	0.185126	0.331170
mae	0.130236	0.151377
mse	0.053554	0.106431
psnr	12.712070	9.729317
rmse	0.231418	0.326238
ssim	0.540464	0.609196
Chukchi Sea	bin_accuracy	0.656515	0.675528
iou	0.126601	0.364351
mae	0.269926	0.300754
mse	0.124306	0.246038
psnr	9.055069	6.089983
rmse	0.352571	0.496022
ssim	0.405798	0.385161
Kara Sea	bin_accuracy	0.801598	0.797711
iou	0.282630	0.412451
mae	0.162785	0.185920
mse	0.070723	0.136968
psnr	11.504373	8.633821
rmse	0.265939	0.370091
ssim	0.604080	0.590542
Laptev Sea	bin_accuracy	0.839829	0.863018
iou	0.387533	0.534633
mae	0.115111	0.122237
mse	0.051770	0.094377
psnr	12.859248	10.251351
rmse	0.227529	0.307208
ssim	0.782073	0.746823
Sea of Japan	bin_accuracy	0.994356	0.989473
iou	0.000000	0.035046
mae	0.013824	0.016332
mse	0.004467	0.009577
psnr	23.499943	20.187490
rmse	0.066835	0.097865
ssim	0.841847	0.879064