Title: Enhancing Transparency in Self-Supervised Learning Through Representer Landmarks and Nyström Approximation

Abstract: While self-supervised learning (SSL) effectively extracts representations from vast amounts of unlabeled data, these models often function as opaque systems, creating a need for domain-specific interpretability. To address this, we present KREPES, a comprehensive framework designed to provide analytical insights into the representations learned by various SSL objectives, such as BYOL, VICReg, and SimCLR. By connecting empirical approximations of the neural tangent kernel with the Representer Theorem, KREPES maps the learned latent space directly to "Representer Landmarks"—the embeddings of pivotal unlabeled training instances. We propose several new metrics, including the "Feature Alignment Gap," "Concept-Conditioned Influence Score," and "Sample-Specific Influence Score," to measure the clarity of these representations. This framework allows for unsupervised auditing of the latent space, uncovering issues such as algorithmic bias in the Adult-1M dataset, where SSL models inadvertently rely on demographic variables as proxies for income. Furthermore, to guarantee scalability for large-scale benchmarks containing over one million samples, such as ImageNet-1K and Adult-1M, KREPES incorporates a novel analytical inference framework based on Nyström approximation.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC