Title: Behavior-Invariant Task Representation Learning with Transformer-based World Models for Offline Meta-Reinforcement Learning

Abstract: Offline meta-reinforcement learning utilizes static datasets to empower agents to generalize across unseen environments, merging the efficiency of offline learning with the adaptability of meta-learning. However, this field struggles with significant challenges related to context and policy distribution shifts. These obstacles impede an agent's ability to adapt to online settings, a problem that intensifies in sparse-reward scenarios. Consequently, agents frequently fall into an inherent pattern dilemma, unable to secure robust generalization. To address these issues, we introduce a novel framework that combines information-theoretic task representation learning with a Transformer-based stochastic world model. This method isolates latent variables that define the task while remaining invariant to the behavior policy, effectively neutralizing context distribution shifts. Furthermore, to manage policy shifts and prevent model exploitation, we implement a conservative value penalty on imagination-based rollouts. This mechanism stops the policy from capitalizing on model inaccuracies while preserving strong adaptation capabilities. Comprehensive evaluations reveal that our approach surpasses current state-of-the-art methods, offering enhanced stability and generalization in out-of-distribution and sparse-reward contexts.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC