Title: SHARP: Sleep-based Hierarchical Accelerated Replay for Long Range Non-Stationary Temporal Pattern Recognition

Abstract:

Capturing long-range, non-stationary temporal patterns continues to pose a significant hurdle for contemporary sequence models, especially within rigid streaming environments. In such scenarios, data streams in sequentially, requiring single-pass processing that precludes simultaneous re-evaluation of historical observations. Conventional architectures, such as transformers and recurrent neural networks (RNNs), face inherent limitations in long-range credit assignment due to constraints like truncated backpropagation through time or fixed input window sizes.

To overcome these bottlenecks, we introduce SHARP (Sleep-based Hierarchical Accelerated Replay). This framework splits temporal learning into two distinct, complementary functions: a pattern-recognition module and a memory module. The latter stores a structured history of past inputs, while the former processes this accumulated memory. By decoupling these tasks, SHARP facilitates resource-efficient adaptation to non-stationary dynamics, removing the computational burden of backpropagating through time for extensive steps.

Drawing inspiration from the accelerated replay of temporally structured memory traces seen in rodents during slow-wave sleep, SHARP integrates offline "sleep" phases. During these periods, memory traces are replayed at an accelerated rate and assimilated into higher-level memory representations, thereby enhancing the retention of long-range context.

We evaluate the framework’s core characteristics through ablation studies and controlled simulations. On benchmark datasets including text8 and PG-19, SHARP outperforms standard recurrent baselines. It successfully maintains next-token predictive accuracy on previously encountered data while simultaneously learning from the live data stream and generalizing to unseen future inputs. These improvements are driven by the model’s hierarchical design, which provides an exponentially expanding effective temporal context while maintaining linear-time computational complexity.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC