Title: Reducing Spurious Correlations Through Memorization-Driven Dataset De-Biasing

Abstract

In many real-world datasets, spurious correlations exist that lack a causal link to the target label. When these non-causal patterns dominate the training set, models frequently latch onto them, resulting in the misclassification of minority samples that do not follow these spurious trends. Although selecting data subsets to better reflect minority groups is a possible remedy, this strategy often necessitates group labels, which are usually unavailable. Moreover, our analysis reveals that standard sample scoring functions employed in invariant subset or coreset selection methods rely heavily on spurious features. Consequently, they struggle to accurately gauge the significance or difficulty of the core, causally relevant features.

To address this, we propose a method to mitigate spurious correlations by creating a two-stage sample scoring function. This approach disentangles the learning dynamics of core and spurious features, assessing their respective difficulties independently. Leveraging this new metric, we introduce an algorithm designed to identify and prioritize informative samples, regardless of whether they exhibit spurious correlations. Comprehensive experiments show that a standard Empirical Risk Minimization (ERM) model trained on our selected subset outperforms current state-of-the-art debiasing techniques, achieving this superior performance using as little as 10% of the original training data.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC