aiice-bench 0.14.0

A standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration

Aiice

---

AIICE is an open-source Python framework designed as a standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration. It provides reproducible pipelines for loading, preprocessing, and evaluating satellite-derived OSI-SAF data, supporting both short- and long-term prediction horizons

Installation

The simplest way to install framework with pip:

pip install aiice-bench

Quickstart

The AIICE class provides a simple interface for loading Arctic ice data, preparing datasets, and benchmarking PyTorch models:

from aiice import AIICE

# Initialize AIICE with a sliding window 
# of past 30 days and forecast of 7 days
aiice = AIICE(
    pre_history_len=30,
    forecast_len=7,
    batch_size=32,
    start="2022-01-01",
    end="2022-12-31"
)

# Define your PyTorch model
model = MyModel()

# Run benchmarking to compute metrics on the dataset
report = aiice.bench(model)
print(report)

Check package doc and see more usage examples. You can also explore the raw dataset and work with it independently via Hugging Face

Leaderboard

The leaderboard reports the mean performance of each model across the evaluation dataset. You can check models' setup in examples.

		baseline_mean	baseline_repeat	conv2d	conv3d
Barents Sea	bin_accuracy	0.874963	0.848936	0.937071	0.891255
iou	0.185126	0.331170	0.647688	0.420801
mae	0.130236	0.151377	0.067575	0.113846
mse	0.053554	0.106431	0.028444	0.064654
psnr	12.712070	9.729317	15.460110	11.894089
rmse	0.231418	0.326238	0.168653	0.254271
ssim	0.540464	0.609196	0.696043	0.618139
Chukchi Sea	bin_accuracy	0.656515	0.675528	0.947459	0.789110
iou	0.126601	0.364351	0.865943	0.585862
mae	0.269926	0.300754	0.069100	0.198657
mse	0.124306	0.246038	0.023475	0.125997
psnr	9.055069	6.089983	16.293947	8.996499
rmse	0.352571	0.496022	0.153215	0.354958
ssim	0.405798	0.385161	0.651510	0.449680
Kara Sea	bin_accuracy	0.801598	0.797711	0.939550	0.844245
iou	0.282630	0.412451	0.785852	0.559398
mae	0.162785	0.185920	0.065702	0.149524
mse	0.070723	0.136968	0.025262	0.092185
psnr	11.504373	8.633821	15.975368	10.358038
rmse	0.265939	0.370091	0.158939	0.303539
ssim	0.604080	0.590542	0.725831	0.589535
Laptev Sea	bin_accuracy	0.839829	0.863018	0.964288	0.897629
iou	0.387533	0.534633	0.859092	0.683309
mae	0.115111	0.122237	0.043340	0.094628
mse	0.051770	0.094377	0.015273	0.066438
psnr	12.859248	10.251351	18.160892	11.784326
rmse	0.227529	0.307208	0.123582	0.257630
ssim	0.782073	0.746823	0.837163	0.802543
Sea of Japan	bin_accuracy	0.994356	0.989473	0.994356	0.995731
iou	0.000000	0.035046	0.000000	0.000000
mae	0.013824	0.016332	0.009841	0.008582
mse	0.004467	0.009577	0.005990	0.004908
psnr	23.499943	20.187490	22.225956	22.945065
rmse	0.066835	0.097865	0.077393	0.069567
ssim	0.841847	0.879064	0.922021	0.919443