temporal_linear_network 0.1.3

Temporal Linear Network

TLN: Temporal Linear Network

This is the original implementation of the paper.

TLN (Temporal Linear Network) is a neural network architecture that extends the capabilities of linear models while maintaining interpretability and computational efficiency. TLN is designed to effectively capture both temporal and feature-wise dependencies in multivariate time series data. Our approach is a variant of TSMixer that maintains strict linearity throughout its architecture. It removes activation functions, introduces specialized kernel initializations, and incorporates dilated convolutions to handle various time scales, all while preserving the linear nature of the model. Unlike transformer-based models that may lose temporal information due to their permutation-invariant nature, TLN explicitly preserves and leverages the temporal structure of the input data. A key innovation of TLN is its ability to compute an equivalent linear model, offering a level of interpretability not found in more complex architectures like TSMixer. This feature allows for seamless conversion between the full TLN model and its linear equivalent, facilitating both training flexibility and inference optimization.

The implementation is made in Keras3 in a backend-agnostic way, to be compatible with TensorFlow, JAX, and Torch. The package also includes implementations of comparison models from the literature, such as CLinear, NLinear, DLinear from "Are Transformers Effective for Time Series Forecasting?" and TSMixer: An All-MLP Architecture for Time Series Forecasting in keras3.

Installation

Install TLN directly from PyPI:

pip install temporal_linear_network

Key Features

• Fully linear architecture that maintains interpretability
• Ability to extract equivalent linear weights for analysis
• Consistent performance across varying sequence lengths
• Compatible with all Keras 3 backends (TensorFlow, JAX, PyTorch)
• Includes implementations of comparison models (CLinear, DLinear, NLinear, TSMixer)

Usage

Here's a basic example demonstrating how to use TLN for time series forecasting:

import keras
from tln import TLN

# Create and configure the model
model = TLN(
    output_len=prediction_horizon,
    output_features=1,
    hidden_layers=2,
    use_convolution=True
)

# Build and compile the model
model.compile(
    optimizer=keras.optimizers.Adam(0.001),
    loss='mean_squared_error',
    jit_compile=True
)

# Train the model
history = model.fit(
    X_train, 
    y_train,
    batch_size=32,
    epochs=100,
    validation_split=0.2
)

# Make predictions
predictions = model.predict(X_test)

# Extract equivalent linear weights for interpretation
weights, bias = model.compute_linear_equivalent_weights()

Advanced Features

Linear Model Conversion

TLN can be converted to its equivalent linear form for analysis:

# Get the equivalent linear model
linear_model = model.get_linear_equivalent_model()

Comparison Models

The package also includes implementations of other linear architectures:

Linear Models from "Are Transformers Effective for Time Series Forecasting?"

from tln import CLinear, NLinear, DLinear

# Classic Linear model
model = keras.Sequential([
    keras.layers.Input(shape=input_shape),
    CLinear(pred_len=prediction_horizon, individual=False),
    keras.layers.Dense(1),  # For multivariate to univariate predictions
    keras.layers.Flatten()
])

# Normalized Linear model
model = keras.Sequential([
    keras.layers.Input(shape=input_shape),
    NLinear(pred_len=prediction_horizon, individual=False),
    keras.layers.Dense(1),  # For multivariate to univariate predictions
    keras.layers.Flatten()
])

# Decomposition Linear model
model = keras.Sequential([
    keras.layers.Input(shape=input_shape),
    DLinear(pred_len=prediction_horizon, individual=False),
    keras.layers.Dense(1),  # For multivariate to univariate predictions
    keras.layers.Flatten()
])

TSMixer Model

from tln import TSMixer

# TSMixer with single block
model = keras.Sequential([
    keras.layers.Input(shape=input_shape),
    TSMixer(
        input_shape=input_shape,
        pred_len=prediction_horizon,
        norm_type='L',
        activation='relu',
        n_block=1,
        dropout=0.0,
        ff_dim=5,
        target_slice=None
    ),
    keras.layers.Dense(1),  # For multivariate to univariate predictions
    keras.layers.Flatten()
])

Please cite our work if you use this repo:

@article{genet2024tln,
  title={A Temporal Linear Network for Time Series Forecasting},
  author={Genet, Remi and Inzirillo, Hugo},
  journal={arXiv preprint arXiv:2410.21448},
  year={2024}
}

Shield:

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.