Title: Signed Dual Attention: Capturing Signed Dependencies in Time Series Forecasting

Abstract

Transformer architectures and their associated attention mechanisms, originally designed for natural language processing, have become foundational components of numerous deep learning models, particularly in the realm of time series forecasting. However, conventional attention mechanisms typically presuppose homophilic interactions, which constrains their capacity to represent datasets exhibiting both positive and negative dependencies, such as time series data. To address this limitation, we propose Signed Dual Attention, a novel formulation capable of identifying both supportive and contrastive relational patterns without incurring additional parameter costs. Drawing inspiration from correlation structures, this method employs a dual message-passing scheme that transmits both positive and negative information within a unified shared block. This design effectively replicates the expressive power of dual-head attention while maintaining parameter efficiency. The proposed module integrates effortlessly into existing frameworks and delivers performance improvements in scenarios necessitating signed relational modeling, thereby paving the way for more expressive and resource-efficient Transformer architectures.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC