aiice-bench 0.15.0

A standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration

Aiice

---

AIICE is an open-source Python framework designed as a standardized benchmark for spatio-temporal forecasting of Arctic sea ice concentration. It provides reproducible pipelines for loading, preprocessing, and evaluating satellite-derived OSI-SAF data, supporting both short- and long-term prediction horizons

Installation

The simplest way to install framework with pip:

pip install aiice-bench

Quickstart

The AIICE class provides a simple interface for loading Arctic ice data, preparing datasets, and benchmarking PyTorch models:

from aiice import AIICE

# Initialize AIICE with a sliding window 
# of past 30 days and forecast of 7 days
aiice = AIICE(
    pre_history_len=30,
    forecast_len=7,
    batch_size=32,
    start="2022-01-01",
    end="2022-12-31"
)

# Define your PyTorch model
model = MyModel()

# Run benchmarking to compute metrics on the dataset
report = aiice.bench(model)
print(report)

Check package doc and see more usage examples. You can also explore the raw dataset and work with it independently via Hugging Face

Leaderboard

The leaderboard reports the mean performance of each model across the evaluation dataset. You can check models' setup in examples.

		baseline_mean	baseline_repeat	conv2d	conv3d	convlstm
Barents Sea	bin_accuracy	0.874963	0.848936	0.937071	0.891255	0.941710
iou	0.185126	0.331170	0.647688	0.420801	0.671497
mae	0.130236	0.151377	0.067575	0.113846	0.057296
mse	0.053554	0.106431	0.028444	0.064654	0.025509
psnr	12.712070	9.729317	15.460110	11.894089	15.948077
rmse	0.231418	0.326238	0.168653	0.254271	0.159578
ssim	0.540464	0.609196	0.696043	0.618139	0.784737
Chukchi Sea	bin_accuracy	0.656515	0.675528	0.947459	0.789110	0.948085
iou	0.126601	0.364351	0.865943	0.585862	0.871389
mae	0.269926	0.300754	0.069100	0.198657	0.061957
mse	0.124306	0.246038	0.023475	0.125997	0.026552
psnr	9.055069	6.089983	16.293947	8.996499	15.796962
rmse	0.352571	0.496022	0.153215	0.354958	0.162596
ssim	0.405798	0.385161	0.651510	0.449680	0.751346
Kara Sea	bin_accuracy	0.801598	0.797711	0.939550	0.844245	0.945934
iou	0.282630	0.412451	0.785852	0.559398	0.805982
mae	0.162785	0.185920	0.065702	0.149524	0.054693
mse	0.070723	0.136968	0.025262	0.092185	0.022224
psnr	11.504373	8.633821	15.975368	10.358038	16.550935
rmse	0.265939	0.370091	0.158939	0.303539	0.148913
ssim	0.604080	0.590542	0.725831	0.589535	0.810979
Laptev Sea	bin_accuracy	0.839829	0.863018	0.964288	0.897629	0.970806
iou	0.387533	0.534633	0.859092	0.683309	0.882638
mae	0.115111	0.122237	0.043340	0.094628	0.030666
mse	0.051770	0.094377	0.015273	0.066438	0.011886
psnr	12.859248	10.251351	18.160892	11.784326	19.400338
rmse	0.227529	0.307208	0.123582	0.257630	0.108099
ssim	0.782073	0.746823	0.837163	0.802543	0.907448
Sea of Japan	bin_accuracy	0.994356	0.989473	0.994356	0.995731	0.995423
iou	0.000000	0.035046	0.000000	0.000000	0.383335
mae	0.013824	0.016332	0.009841	0.008582	0.006749
mse	0.004467	0.009577	0.005990	0.004908	0.002619
psnr	23.499943	20.187490	22.225956	22.945065	25.774397
rmse	0.066835	0.097865	0.077393	0.069567	0.051133
ssim	0.841847	0.879064	0.922021	0.919443	0.923896