Title: Flexible Online Representation Learning Based on Similarity Matching

Abstract:

Sparse, high-dimensional representations play a crucial role in revealing complex, non-trivial structures during the unsupervised exploration of data. These representations are particularly effective at managing the dense connectivity found in graphs, which is a common challenge in community detection tasks. Beyond graph analysis, such representations offer additional capabilities, including manifold tiling and feature learning.

Traditional algorithms often struggle with these tasks because they either optimize within the space of completely positive matrices—which is computationally intractable—or they relax the problem into the space of doubly nonnegative matrices. The latter approach scales with sample size in a manner that makes it impractical for large-scale datasets. Furthermore, some existing methods enforce row sum constraints, such as double stochasticity. While these constraints offer the benefit of shift-invariance in the context of manifold tiling, they necessitate complex online learning rules to satisfy the requirements of the output similarity matrices.

To address these challenges, we introduce a versatile, biologically plausible online learning algorithm. This method is capable of learning sparse, shift-invariant representations that adapt to various data structures, making it suitable for applications such as clustering, manifold tiling, and sparse coding.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC