Title: MIC: Maximizing Informational Capacity in Adaptive Representations via Isotropic Subspace Alignment

Abstract:

While multi-scale representation learning facilitates the creation of elastic-dimensional embeddings, nested subspaces are frequently plagued by dimensional redundancy and spectral collapse. To resolve these challenges, we present MIC, a novel framework that refines the geometric structure of multi-granular embeddings by employing isotropic subspace alignment. The MIC architecture integrates Soft Collapse Regularization (SCR) to reduce redundancy between prefix and residual subspaces through cross-correlation penalties. Simultaneously, it utilizes Spectral Isotropy Regularization (SIR) to enforce hyper-spherical uniformity within low-dimensional prefixes. By harmonizing these approaches through a self-distillation objective, MIC produces semantically dense representations that retain robust discriminative capabilities. Experimental results indicate that MIC substantially surpasses standard baselines, with particularly notable improvements in high-compression settings where preserving informational capacity is of paramount importance.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC