Conforme

Conforme is a conformal forecasting library for panels of time series. It makes point forecasts at many origins, reconciles them over a hierarchy, and calibrates bands and bounds from out-of-sample residuals.

Install

Install conforme from PyPI:

pip install conforme
# or
uv add conforme

The vendor model adapters need an extra: stats (statsforecast), ml (mlforecast), or neural (neuralforecast).

pip install "conforme[stats]"

Quick start

import numpy as np
import pandas as pd

from conforme import Absolute, BottomUp, Hierarchy, LeadTime, Panel, SeasonalNaive, Signed
from conforme import SplitQuantile, Step, replay, rolling_forecasts
from conforme.metrics import coverage

rng = np.random.default_rng(0)
series = np.array(["a", "b", "c"])
periods = pd.date_range("2024-01-01", periods=120, freq="D")
panel = Panel(series, periods, rng.poisson(5.0, (3, 120)), "D")
hierarchy = Hierarchy.from_attributes(
    series, pd.DataFrame({"group": ["x", "x", "y"]}, index=series)
)

origins = np.arange(60, 113)  # each origin is the index of the last observed period
run = rolling_forecasts(panel, hierarchy, SeasonalNaive(7), BottomUp(hierarchy), origins, 7)
actuals = hierarchy.aggregate(panel.values)

calibrator = SplitQuantile(0.9, window=28)
bands = replay(run, actuals, target=Step(), score=Absolute(), calibrator=calibrator)
bound = replay(run, actuals, target=LeadTime(7), score=Signed(), calibrator=calibrator)
print(coverage(bands.target, bands.lower, bands.upper))

Concepts

Shapes use these axes: B bottom series, T periods, N nodes with bottoms first, S forecast series (B for BottomUp, N otherwise), O origins, H forecast steps, and C target columns (H for Step, 1 for LeadTime).

Two rules hold everywhere. A model reads only its window, so a later value cannot change a point unless a covariate declares it known ahead. A calibrator sees a score only once its target is known, and before the origin that knows it issues.

Architecture has the shapes and the modules.

Write a calibrator

A calibrator owns its target, a level or a loss, and is three functions of an explicit state. conforme.online handles the origins, the delays, and the indexing. A new method is one file in conforme/conformal/calibrators/ that imports only calibrators.base, calibrators.ranks, and conformal.losses. This is the whole of a quantile tracker:

import numpy as np

from conforme.conformal.calibrators.base import Calibrator


class Tracker(Calibrator):
    def __init__(self, level, lr):
        self.level = level
        self.lr = lr

    def initial_state(self, n_nodes, n_columns):
        return {"q": np.zeros((n_nodes, n_columns))}

    def update(self, state, feedback):
        q = state["q"].copy()
        for row, column in enumerate(feedback.column):  # one row = one origin, one column
            miss = feedback.scores[row] > feedback.issued[row]
            q[:, column] += self.lr * (miss - (1 - self.level))
        return {"q": q}

    def threshold(self, state):
        return state["q"].astype(np.float32)

Keep the state in numpy arrays. Then a product can save it after each origin with conforme.online.state.flatten and continue from it.

Models

from lightgbm import LGBMRegressor
from conforme import BottomUp, Covariate, rolling_forecasts
from conforme.forecast.models.mlforecast import MLForecast

model = MLForecast(
    LGBMRegressor(), lags=[7, 14, 28], features=["price"], fit_periods=365, lookback=84
)
price = Covariate(prices, known_ahead=True, aggregate="mean")  # [B, T + H]
run = rolling_forecasts(
    panel,
    hierarchy,
    model,
    BottomUp(hierarchy),
    origins,
    28,
    refit_every=7,
    covariates={"price": price},
)

Benchmarks

bench/ is a folder of scripts. They load benchmark data from a directory, run benchmark protocols, and compare calibrators. They import the installed conforme. Raw data is not in Git.

vn2_conformal.py runs the whole VN2 path in about one second:

uv run python bench/vn2_conformal.py data/vn2

Documents

• Architecture: modules, array contracts, and costs.
• Semantics: the calibration rules.

Development

uv sync --group dev
uv run pytest
uv run ruff check .
uv run ruff format --check .
uv run ty check src/
uv build --no-sources

See Contributing.

License

MIT