Title: COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space

Abstract: Unsupervised skill discovery (USD) seeks to acquire a variety of behaviors without relying on reward functions; however, this approach often yields actions that are either irrelevant to the task or dangerous, largely due to uniform exploration strategies. Guided skill discovery (GSD) mitigates these problems by integrating human intent to steer exploration toward meaningful areas. Despite this, current GSD techniques generally depend on training supplementary guidance models and utilizing pre-set rules or expert demonstrations, which proves inefficient when dealing with sparse, online human feedback. To address these limitations, we introduce COLLIE, a GSD framework that utilizes dense unsupervised data to build a semantically coherent latent space for skills. The robust structure of this latent space ensures that guidance remains reliable even with limited online feedback. Furthermore, the inherent semantic coherence allows for the creation of guidance signals without any additional training, thereby removing the necessity for models beyond the skill learner itself. Our theoretical analysis supports the efficacy of this training-free guidance mechanism. Experimental results across various state-based and pixel-based environments demonstrate that COLLIE successfully learns diverse, human-aligned skills, prevents hazardous actions, and delivers enhanced downstream performance with minimal human input.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC