Title: Why Not Hyperparameter-Friendly Optimisation? A Monotonic Adaptive Norm Rescaling Approach For Long-Tailed Recognition

Abstract:

Deep learning faces substantial hurdles in the realm of long-tailed recognition. One promising avenue involves a two-stage decoupling paradigm that disentangles representation learning from the subsequent retraining of the classifier. Within the classifier retraining phase, adaptive norm rescaling has emerged as a widely adopted technique. This method typically modifies the norms of per-class weights through parameter regularization; however, this process inevitably brings hyperparameters into the mix. Numerous studies have highlighted that long-tailed recognition tasks are highly sensitive to these hyperparameters, with their configuration playing a critical role in determining overall performance.

In this work, we first offer a perspective grounded in class-conditional distributions to justify the use of norm rescaling techniques. Building on this foundation, we introduce a straightforward yet potent method named Self-Adaptive Monotonic Normalization (SAMN). Unlike conventional approaches, SAMN eliminates the reliance on parameter regularization. Instead, it ensures monotonicity in per-class weight norms by employing the Pool Adjacent Violators Algorithm, thereby rendering the technique hyperparameter-friendly. As a versatile strategy, SAMN can be effortlessly integrated with existing methods to further elevate performance. Our experiments across various benchmark datasets reveal that SAMN markedly improves long-tailed recognition outcomes, frequently securing state-of-the-art results.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC