Title: ReaLM: Residual Quantization Bridging Knowledge Graph Embeddings and Large Language Models

Abstract:

Large Language Models (LLMs) have recently established themselves as a potent approach for Knowledge Graph Completion (KGC), surpassing conventional embedding techniques with their superior reasoning and generalization abilities. Nevertheless, current LLM-centric strategies frequently fail to leverage structured semantic representations effectively. This limitation stems from a fundamental misalignment between the continuous embedding space of pre-trained KG models and the discrete token space inherent to LLMs, a gap that impedes efficient semantic transfer and caps performance potential.

To overcome this hurdle, we introduce ReaLM, an innovative framework designed to connect KG embeddings with LLM tokenization via residual vector quantization. By converting pre-trained KG embeddings into compact code sequences and embedding them as learnable tokens within the LLM’s vocabulary, ReaLM facilitates the seamless integration of symbolic and contextual knowledge. Additionally, the method employs ontology-guided class constraints to maintain semantic consistency, thereby enhancing entity predictions through class-level compatibility analysis. Comprehensive evaluations on two prominent benchmark datasets reveal that ReaLM delivers state-of-the-art results, validating its capacity to align structured knowledge with large-scale language models.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC