intermittent-alignment-error 0.1.2

Timing-aware metric for intermittent demand forecasting

pip install intermittent_alignment_error

Intermittent Alignment Error
Timing-aware evaluation for intermittent demand forecasts

This repository contains the reference implementation of Intermittent Alignment Error (IAE), a metric for evaluating intermittent demand forecasts when both timing and magnitude matter.

IAE is most useful when forecasts are expressed in the same unit as the original time-series, with zero-demand periods and non-zero demand events across a forecasting horizon. It is designed for point forecasts and is especially useful when comparing models across heterogeneous intermittent time-series.

For a quick start, see the example.ipynb notebook in this repository.

---

Examples

The examples below illustrate how the metric behaves under different types of forecasting errors.

Early and late forecasts

Forecast shifted early. The forecast is close in time to the ground truth and arrives before demand occurs, which tends to keep the in-time error low.	Forecast shifted late. The forecast may still be close in time to the ground truth, but demand is not covered before it occurs, which increases the in-time error.

Metric components

Recall and precision error. Recall error measures how well actual demand events are captured by the forecast. Precision error measures how well forecasted demand is supported by actual demand.	Mass and in-time error. Mass error measures whether the total demand over the horizon is correct. In-time error measures whether enough demand has been forecasted before actual demand occurs.

Standard timing-aware setting

This figure shows how the metric behaves under different combinations of timing and magnitude errors for three different ADI scenarios. Each matrix visualises the average metric score when forecasts are shifted earlier or later in time and scaled up or down in demand size, illustrating how the standard IAE setting responds to different types of forecasting errors across varying levels of intermittency.

---

Demand rate setting

Some forecasting methods, such as Croston-style demand rate forecasts, are not intended to predict exact event timing. In that case, the metric can be configured to focus only on mass and in-time.

Demand rate forecast example

Demand rate metric matrix

To use the demand rate setting, turn off recall and precision by setting their weights to zero:

metric_params = {
    "recall_weight": 0.0,
    "precision_weight": 0.0,
    "mass_weight": 1.0,
    "in_time_weight": 1.0,
    "p": 2.0,
    "in_time_relevance_mode": "linear",
}

This configuration is appropriate when the forecast is intended to represent a rate or cumulative coverage over the horizon rather than exact local alignment of demand events.

---

The version of the metric used in the licentiate thesis Decision Support through Global Demand Forecasting is preserved in the licentiate_thesis branch. The main branch contains the current version of the implementation.

License

This work is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).

Please cite the corresponding paper when using this metric in academic work.