Title: FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting

Original: arXiv:2606.01306v1 Announce Type: new Abstract: While Transformer-based architectures have established themselves as a dominant paradigm in Multivariate Time Series Forecasting (MTSF), their core self-attention mechanism inherently functions as a low-pass filter, systematically smoothing out high-frequency signals vital for sharp local changes. Recent advancements have increasingly incorporated frequency-domain operations to address this bias, however, most existing designs rely on fixed spectral bases and apply sequence-wise (uniform) modulation, implicitly assuming a time-invariant frequency response. This overlooks a key property of real-world series that their spectral characteristics often evolve over time, making uniform modulation insufficient for capturing fine-grained temporal dynamics. To tackle these limitations, we propose FAiT, a Frequency-Aware inverted Transformer. Specifically, FAiT rectifies the spectral bias internally through Inverted Attention, which interprets the attention map as a learnable low-pass operator and constructs a dedicated complementary high-pass branch by inverting the attention matrix to recover attenuated transient signals. Furthermore, FAiT introduces Dynamic Temporal-Frequency Modulation (DTFM), which synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands, enabling fine-grained control over evolving multi-scale patterns. Extensive experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines, while maintaining computational efficiency.

Rewrite: Despite the dominance of Transformer-based models in Multivariate Time Series Forecasting (MTSF), their fundamental self-attention mechanism acts as a low-pass filter. This tendency systematically dampens high-frequency signals, which are crucial for detecting abrupt local variations. Although recent studies have attempted to mitigate this bias by integrating frequency-domain techniques, most approaches utilize static spectral bases and apply uniform, sequence-level modulation. This methodology presumes a time-invariant frequency response, thereby neglecting the reality that real-world time series often exhibit evolving spectral traits. Consequently, static modulation proves inadequate for capturing intricate temporal dynamics. To overcome these constraints, we introduce FAiT, an inverted Transformer architecture designed with frequency awareness. FAiT addresses internal spectral bias via Inverted Attention. By treating the attention map as a trainable low-pass operator, the model generates a complementary high-pass branch through the inversion of the attention matrix, effectively restoring weakened transient signals. Additionally, FAiT employs Dynamic Temporal-Frequency Modulation (DTFM). This component generates instance-specific weights to dynamically adjust the energy levels of spectral sub-bands, allowing for precise management of changing multi-scale patterns. Comprehensive evaluations on standard benchmarks reveal that FAiT not only surpasses current state-of-the-art Transformer and frequency-enhanced baselines but also does so with high computational efficiency.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC