Panelformer 0.2.1

A transformer-based architecture for robust panel time series forecasting, extending the Temporal Fusion Transformer with multi-scale decomposition, Segment-wise Attention, cross-entity attention, and adaptive trend-seasonal modeling.

Panelformer

Panelformer is a transformer-based deep learning model designed for accurate and scalable panel time series forecasting. Built on top of the Temporal Fusion Transformer (TFT), Panelformer introduces several innovations to address the limitations of existing models when applied to heterogeneous panel datasets.

---

🔑 Keywords

Panel time series, Temporal Fusion Transformer, Transformer architecture, Forecasting, Panelformer

---

🔍 Overview

Panel time series involve multiple entities (e.g., countries, products) observed over time — requiring models to handle both temporal and cross-sectional complexity. Panelformer enhances forecasting accuracy across diverse panel structures by integrating the following features:

• Segment-wise Attention: Reduces complexity and captures local patterns in long sequences.
• Multi-Scale Series Decomposition: Separates trend and seasonal components using learnable moving averages.
• Cross-Entity Attention: Models dependencies between different panel entities.
• Parallel Trend-Seasonal Paths: Dedicated processing for distinct temporal dynamics.
• Adaptive Component Weighting: Learns to prioritize seasonal vs. trend features dynamically.

🚀 Key Results

Evaluated on 11 real-world datasets spanning economics, energy, climate, and health domains, Panelformer achieved:

• 7.99% average MAPE improvement over baseline TFT
• Robust performance on balanced/unbalanced, short/long, micro/macro panels
• Significant accuracy gains on high-frequency datasets like foreign exchange, surface temperature, and electricity

📦 Installation

pip install panelformer

---

🛠️ How to Use

Using Panelformer involves four main steps:

1. Data Preparation

Prepare your panel dataset with columns for entity IDs, timestamps, and target values. Convert your date column to datetime format and create a sequential time index for modeling.

2. Dataset Creation

Use the pytorch_forecasting library’s TimeSeriesDataSet class to define training, validation, and test datasets. This includes specifying encoder and decoder lengths, grouping by entity, and applying normalization.

---

🏋️ Step 3: Train the Model

Instantiate the Panelformer model using the prepared dataset, configure hyperparameters (like learning rate, hidden sizes, attention heads), and train with a PyTorch Lightning Trainer for efficient training management.

import lightning.pytorch as pl
from lightning.pytorch.callbacks import EarlyStopping, LearningRateMonitor
from lightning.pytorch.loggers import TensorBoardLogger
from pytorch_forecasting.metrics import QuantileLoss

early_stop_callback = EarlyStopping(monitor="val_loss", min_delta=1e-10, patience=5, verbose=False, mode="min")
lr_logger = LearningRateMonitor()
logger = TensorBoardLogger("lightning_logs")

trainer = pl.Trainer(
    max_epochs = #Integer
    enable_model_summary=True,
    gradient_clip_val=0.1,
    callbacks=[lr_logger, early_stop_callback],
    logger=logger,
)

model = Panelformer.from_dataset(
    training,
    learning_rate=0.001,
    hidden_size=16,
    attention_head_size=2,
    dropout=0.1,
    segment_size=4,
    decomposition_kernel_sizes=[3, 7, 15, 31],
    trend_processing_layers=2,
    use_cross_series_attention=True,
    adaptive_trend_weight=True,
    hidden_continuous_size=8,
    loss=QuantileLoss()
)

trainer.fit(model, train_dataloaders=train_dataloader, val_dataloaders=val_dataloader)

---

📈 Step 4: Make Predictions

Use the trained model to predict on new data by passing a test dataloader, retrieving forecasts for evaluation or downstream use.

predictions = model.predict(test_dataloader, mode="raw", return_x=True)

---

---

🛠 License

Released under the MIT License. Built on top of PyTorch Forecasting.

---