Title: The Necessity of Extended Context Windows in Time Series Modeling

Abstract:

Contemporary deep learning architectures designed for forecasting multiple time series are increasingly utilizing extensive observation windows. While the prevailing assumption is that larger windows primarily serve to capture long-range dependencies, there has been a notable lack of comprehensive discourse regarding how global forecasting models effectively utilize these input observations. This study posits that forecasting groups of time series is driven by two distinct objectives: (i) Generative Process Identification (GPI), which involves deducing the specific mechanism generating the input sequence, and (ii) Conditional Forecasting (CF), which entails predicting future values based on the observed data.

Viewed through this dual-lens framework, optimal forecasts can be understood as a weighted average across plausible data-generating processes, with weights determined by their likelihood relative to the input window. This perspective offers an alternative rationale for the efficacy of long context windows: they serve to diminish uncertainty regarding the specific process responsible for generating the time series during the operational phase. We demonstrate mathematically that even for processes characterized by a memory length of $P$, an input window exceeding $P$ is strictly required to attain the minimum possible error. Furthermore, we illustrate that separating GPI from CF enhances computational scalability while maintaining predictive accuracy. Our findings, validated through experiments on both synthetic and real-world datasets, underscore the importance of these insights in the development of forecasting architectures.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC