Title: Graph Structure as Inherent Regularization: Reexamining Vector Quantization in Graph Representation Learning

Abstract

Vector Quantization (VQ) has recently gained traction as a viable method for generating compressed, discrete representations of graph-structured data. Nevertheless, a critical obstacle known as "codebook collapse" has received insufficient attention within the graph domain, thereby constraining both the expressive power and generalization capabilities of graph tokens.

In this study, we conduct an empirical analysis revealing that codebook collapse is a persistent phenomenon when VQ is trained concurrently with Graph Neural Networks (GNNs) for graph reconstruction tasks. This issue persists even when employing mitigation techniques originally designed for vision or language models. We further diagnose the root causes of this collapse from both data-centric and optimization standpoints, demonstrating that the phenomenon is linked to intrinsic properties of graph data, such as feature redundancy and connectivity density. Additionally, the training dynamics inherent to deterministic hard assignments exacerbate the problem.

To overcome these limitations, we introduce RGVQ, a novel framework that leverages graph topology and feature similarity as explicit regularization mechanisms. This approach aims to boost codebook utilization and foster greater token diversity. RGVQ employs Gumbel-Softmax reparameterization to facilitate soft assignments, which guarantees that all codewords receive gradient updates. Furthermore, it integrates a structure-aware contrastive regularization term that discourages the assignment of identical tokens to dissimilar node pairs. Our extensive experimental results indicate that RGVQ significantly enhances codebook utilization and consistently elevates the performance of leading graph VQ backbones across various downstream tasks. Consequently, this method enables the creation of graph token representations that are both more expressive and more transferable.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC