Title: Can Local Learning Match Self-Supervised Backpropagation?

Abstract: Although end-to-end self-supervised learning driven by backpropagation (global BP-SSL) is foundational to contemporary AI training, theoretical frameworks for local self-supervised learning (local-SSL) have faced significant challenges in generating effective representations within deep neural networks. To bridge the gap between global and local methodologies, we first formulate a theory for deep linear networks, pinpointing the specific conditions under which local-SSL algorithms—such as Forward-forward or CLAPP—can execute weight updates identical to those of global BP-SSL. Leveraging these theoretical findings, we introduce new variants of local-SSL algorithms designed to approximate global BP-SSL in deep, non-linear convolutional neural networks. Our results indicate that variants enhancing the alignment between local-SSL and global BP-SSL gradient updates also deliver superior performance on image datasets, including CIFAR-10, STL-10, and Tiny ImageNet. Notably, the most effective local-SSL rule utilizing the CLAPP loss function achieves performance parity with comparable global BP-SSL models employing InfoNCE or CPC-like loss functions, while simultaneously surpassing existing state-of-the-art results for local SSL on these benchmarks.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC