Title: Extending Gibbs and Langevin Monte Carlo Algorithms to the Interpolation Regime

Abstract: This study establishes data-dependent bounds for the anticipated error of the Gibbs algorithm within the overparameterized interpolation framework. In this context, algorithms achieve low training errors even on nonsensical data, such as randomly labeled classification tasks. The findings indicate that generalization capabilities in the low-temperature phase are already reflected by minimal training errors observed in the noisier, high-temperature phase. Furthermore, these bounds remain robust when approximated using Langevin Monte Carlo methods. This theoretical insight drives the development of a computational algorithm designed to estimate these bounds. When applied to the MNIST, CIFAR-10, and SVHN datasets, the approach delivers precise, nontrivial predictions of test error for genuinely labeled data, while simultaneously ensuring a valid upper bound on the test error for randomly labeled datasets.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC