How Good Can Linear Models Be for Time-Series Forecasting?
Lang Huang, Jinglue Xu, Luke Darlow
Read on arXiv →
Key claim
Preprocessing optimizations outperform larger models in time-series forecasting.
In plain English
Imagine you're trying to predict future values in a time series, like stock prices or weather patterns. Traditionally, many researchers have believed that using larger and more complex models, like specialized transformers, is the best way to improve accuracy. However, this approach can be costly and may not always yield the best results. For instance, larger models can overfit to the data, meaning they perform well on training data but poorly on new, unseen data. This is what's called overfitting, and it can lead to disappointing performance in real-world applications.
In this paper, the authors take a different approach. They argue that instead of just scaling up models, we should focus on optimizing how we preprocess the data. They use Ridge regression, a simpler model with clear and interpretable parameters, to explore various preprocessing techniques. By adjusting factors like how much historical data to consider and how to normalize the data, they discover that these tweaks can lead to better forecasting results. For example, they find that the optimal amount of historical data to use can vary significantly depending on the specific time series being analyzed, which challenges the common belief that more history is always better.
The authors also reveal that normalizing data in a more targeted way, rather than using all available data, often leads to better performance. Their findings show that different series within the same dataset may require different preprocessing strategies, which is a nuanced understanding that larger models might overlook. Overall, their optimized methods outperform traditional linear models and even more complex architectures like Transformers and CNNs on most benchmarks. This means that for builders working on time-series forecasting, focusing on preprocessing could be a more efficient and effective strategy than simply scaling up model size.
The paper introduces a new perspective on time-series forecasting by emphasizing preprocessing over model scaling.
The findings are supported by extensive experimentation across multiple benchmarks, demonstrating clear improvements over existing methods.
Deep reliability assessment
The methodology supports the narrower claim that a heavily tuned Ridge pipeline can be very competitive on standard long-horizon forecasting benchmarks under MSE, especially when context length, normalization window, regularization, and augmentation are searched per dataset/horizon/series. The broader claim that preprocessing closes most of the gap versus scaling is less secure because nonlinear baselines do not appear to receive the same hyperparameter-search budget and the evidence is limited to common academic benchmarks.
Reproducibility
Partially reproducible in principle: the paper uses eight standard time-series benchmarks, reports the search protocol as 20 Optuna trials per cell with 3-fold cross-validation, and says it will release SearchCast, but the provided text only shows a placeholder '(URL)' and no actual repository link.
Key figure
Figure 1 visualizes how the optimal context window length found by hyperparameter search differs across horizons and datasets, showing that longer forecast horizons do not always require longer historical lookback.