**Title: G2LoRA: A Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs

Abstract:

The "LLM-as-Aligner" approach has become a dominant pre-training strategy for Text-Attributed Graphs (TAGs), utilizing CLIP-style contrastive learning to map graph and text modalities into a unified embedding space. Despite their success in isolated downstream tasks, we identify that these models suffer from significant catastrophic forgetting when subjected to sequential fine-tuning on streaming tasks. While parameter-efficient fine-tuning offers some mitigation against forgetting, it fails to adequately address task interference or facilitate effective knowledge transfer.

This study investigates graph continual learning for LLM-as-Aligner models within the context of TAGs, aiming to minimize interference while enhancing positive transfer across tasks. This scenario presents two primary challenges: first, diverse downstream tasks create shifting optimization goals that complicate unified fine-tuning; second, the graph and text encoders display varying sensitivities to adaptation, meaning uncoordinated updates can lead to misalignment.

To tackle these issues, we introduce G2LoRA, a continual learning framework designed specifically for TAGs. G2LoRA consolidates node-, link-, and graph-level tasks into a single graph-text alignment objective, ensuring consistent optimization across incremental modes involving domains, classes, or tasks. To balance forward and backward knowledge flow by reducing task interference and fostering positive transfer, G2LoRA employs category-aware gradient projection within structured subspaces. This mechanism resolves conflicting updates and allows for conditional backward transfer. Additionally, to prevent cross-modal drift, the framework incorporates gradient magnitude modulation to synchronize the update rates of graph and text encoders. Comprehensive experiments on benchmark datasets show that G2LoRA consistently surpasses strong baselines across various backbone architectures, delivering superior continual performance and transferability.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC