Title: Optimizing Online Model Selection for Agility: Mitigating Adaptation Delays Through Protected High Learning Rates

Abstract: In non-stationary settings, preserving predictive precision necessitates autonomous online model selection to adjust to unforeseen distributional shifts. Nevertheless, current algorithms that operate without manual tuning are constrained by a core dilemma between stability and responsiveness. To satisfy dynamic regret bounds, these methods are forced to cap learning rates at small, fixed values (such as $O(1)$). Consequently, this limitation results in substantial delays when adapting to sudden changes. We introduce an innovative optimistic online mirror descent framework that employs safeguarded learning rates reaching up to $\Theta(T)$, with $T$ representing the total number of rounds. The primary technical advancement is a post-hoc penalty system that tracks unstable updates and discards learning rates that would generate excessive regret, thereby removing the necessity for restrictive pre-set constraints. Our analysis confirms that the total penalty accumulates at a rate of $O(\log T)$, enabling the algorithm to achieve near-optimal worst-case performance while delivering enhanced results in favorable conditions. Testing across eleven real-world datasets and three synthetic ones reveals that our method cuts adaptation lag from hundreds of rounds down to just a few, consistently surpassing tuning-free baseline models.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC