aiice-bench 0.13.0

A standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration

Aiice

---

AIICE is an open-source Python framework designed as a standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration. It provides reproducible pipelines for loading, preprocessing, and evaluating satellite-derived OSI-SAF data, supporting both short- and long-term prediction horizons

Installation

The simplest way to install framework with pip:

pip install aiice-bench

Quickstart

The AIICE class provides a simple interface for loading Arctic ice data, preparing datasets, and benchmarking PyTorch models:

from aiice import AIICE

# Initialize AIICE with a sliding window 
# of past 30 days and forecast of 7 days
aiice = AIICE(
    pre_history_len=30,
    forecast_len=7,
    batch_size=32,
    start="2022-01-01",
    end="2022-12-31"
)

# Define your PyTorch model
model = MyModel()

# Run benchmarking to compute metrics on the dataset
report = aiice.bench(model)
print(report)

Check package doc and see more usage examples. You can also explore the raw dataset and work with it independently via Hugging Face

Leaderboard

The leaderboard reports the mean performance of each model across the evaluation dataset. You can check models' setup in examples.

		baseline_mean	baseline_repeat	conv2d
Barents Sea	bin_accuracy	0.874963	0.848936	0.937071
iou	0.185126	0.331170	0.647688
mae	0.130236	0.151377	0.067575
mse	0.053554	0.106431	0.028444
psnr	12.712070	9.729317	15.460110
rmse	0.231418	0.326238	0.168653
ssim	0.540464	0.609196	0.696043
Chukchi Sea	bin_accuracy	0.656515	0.675528	0.947459
iou	0.126601	0.364351	0.865943
mae	0.269926	0.300754	0.069100
mse	0.124306	0.246038	0.023475
psnr	9.055069	6.089983	16.293947
rmse	0.352571	0.496022	0.153215
ssim	0.405798	0.385161	0.651510
Kara Sea	bin_accuracy	0.801598	0.797711	0.939550
iou	0.282630	0.412451	0.785852
mae	0.162785	0.185920	0.065702
mse	0.070723	0.136968	0.025262
psnr	11.504373	8.633821	15.975368
rmse	0.265939	0.370091	0.158939
ssim	0.604080	0.590542	0.725831
Laptev Sea	bin_accuracy	0.839829	0.863018	0.964288
iou	0.387533	0.534633	0.859092
mae	0.115111	0.122237	0.043340
mse	0.051770	0.094377	0.015273
psnr	12.859248	10.251351	18.160892
rmse	0.227529	0.307208	0.123582
ssim	0.782073	0.746823	0.837163
Sea of Japan	bin_accuracy	0.994356	0.989473	0.994356
iou	0.000000	0.035046	0.000000
mae	0.013824	0.016332	0.009841
mse	0.004467	0.009577	0.005990
psnr	23.499943	20.187490	22.225956
rmse	0.066835	0.097865	0.077393
ssim	0.841847	0.879064	0.922021