Title: Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging

Abstract:

While entropy minimization (EM) serves as the primary objective for test-time adaptation, the phenomenon of model collapse remains insufficiently explained. This study reveals that distribution shifts can lead to the merging of feature clusters associated with distinct classes within the model’s representation space, even as the decision boundary stays unchanged. This process generates a systematic distortion in the predicted class distribution, which we term "prediction bias." Specifically, prediction bias describes a scenario where certain classes are overrepresented while others are suppressed in the output distribution. We demonstrate that entropy minimization exacerbates this bias by compressing existing clusters, thereby reinforcing incorrect groupings until predictions degenerate into a trivial solution.

To address the critical nature of this bias, we introduce Distribution Shift Bias Reduction (DSBR), a corrective objective designed to counteract this specific failure mode. DSBR works by equalizing the contribution of each predicted class to the unsupervised entropy minimization loss. To investigate this failure mechanism, we established appropriate adaptation settings using four medical imaging datasets and further tested our approach on ImageNet-C. Our findings indicate that DSBR consistently stabilizes test-time adaptation, prevents model collapse, and achieves performance comparable to or superior than state-of-the-art methods. Notably, DSBR functions exclusively at the test-time stage.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC