Title: Returning to Plato’s Cave: A Scalable Analysis of Cross-Modal Representational Convergence

Abstract: The Platonic Representation Hypothesis posits that artificial neural networks, despite being trained on distinct data modalities such as text and images, will eventually align and converge on a singular, shared representation of reality. If this hypothesis holds, it raises profound questions regarding the necessity of specific modality choices. However, our analysis demonstrates that the empirical support for this claim is precarious and heavily contingent upon the evaluation methodology. When alignment is assessed via mutual nearest neighbors on limited datasets (approximately 1,000 samples), results appear positive; yet, this alignment deteriorates significantly as the dataset expands to include millions of samples. This degradation pattern persists across other modality pairs, including text-audio and text-video. Furthermore, the residual alignment observed between model representations corresponds to broad semantic similarities rather than stable, fine-grained structural matches. Additionally, the evaluations conducted by Huh et al. rely on a rigid one-to-one image-caption framework, a constraint that fails in more realistic many-to-many scenarios and consequently lowers the measured degree of alignment. We also observe that the previously reported trend—where larger language models increasingly align with visual models—does not extend to more recent architectures. Consequently, our results indicate that the existing evidence for cross-modal representational convergence is substantially weaker than subsequent literature has assumed. It appears that models trained on different modalities may indeed acquire equally rich world models, but these representations are not identical.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC