Title: A Dead End: Unpacking the Phenomenon of Deep Spurious Regression

Abstract

In practical regression scenarios, models frequently encounter "shortcuts"—features that show a deceptive correlation with continuous targets during training but prove unstable when distributional shifts occur during deployment. Relying on these spurious correlations can lead to catastrophic failures in testing environments. While previous research on spurious correlations has largely concentrated on classification tasks—where labels are categorical and distinct groups are clearly defined—many real-world applications demand continuous prediction. In such contexts, rigid label boundaries or discrete group-label pairings are absent.

To address this, we introduce the concept of Deep Spurious Regression (DSR), which involves learning from regression data characterized by attribute-label confounding. The goal is to manage continuous spurious correlations and ensure the model generalizes effectively to all possible attribute-label combinations at test time. Acknowledging the fundamental differences between shortcuts in classification and regression, our approach leverages the similarity among spurious attributes within both label and feature spaces. This method accounts for proximate targets and related groups, simultaneously calibrating the distributions of labels and learned features across various attributes.

We conducted extensive experiments on standard real-world DSR datasets covering computer vision, environmental sensing, and regression tasks in large language models (LLMs). The results confirm the superior performance of our proposed strategies. This study addresses a critical gap in existing benchmarks and methodologies for investigating spurious correlations in the realm of continuous prediction.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC