Title: Investigating How Label Encoding Influences Neural Collapse

Neural collapse represents a distinct structural characteristic observed in the final hidden layer activations of neural network classifiers once they have achieved zero classification error. This study examines how label encoding impacts this phenomenon, utilizing an unrestricted feature model trained with a mean squared error loss function.

Our findings indicate that when dealing with balanced datasets and one-hot encoded labels, the uncentered mean features linked to each class shift from a simplex equiangular tight frame to an orthogonal frame as the bias regularization coefficient for the final classifier is increased. This resulting configuration mirrors the orthogonal frame structure inherent to one-hot encoded labels.

Additionally, for any arbitrary encoding scheme, we demonstrate that the bias term in the final classifier functions to center the labels, thereby offsetting the gap between the origin and the global mean of the labels. The paper also explores how encoding affects other properties associated with neural collapse.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC