Title: SUSD: Structured Unsupervised Skill Discovery through State Factorization

Abstract:

Unsupervised Skill Discovery (USD) seeks to autonomously acquire a varied repertoire of skills without the need for external reward signals. While maximizing Mutual Information (MI) between latent skill variables and environmental states is a prevalent USD strategy, these methods often converge on simple, static behaviors due to their invariance characteristics. This tendency restricts the identification of dynamic, task-critical actions. Although Distance-Maximizing Skill Discovery (DSD) attempts to foster more dynamic capabilities by utilizing state-space distances, it still fails to ensure a comprehensive skill set that addresses all controllable factors or entities within the environment.

To address these limitations, we propose SUSD, a new framework that exploits the compositional nature of environments by decomposing the state space into independent components, such as distinct objects or controllable entities. By assigning separate skill variables to each factor, SUSD allows for more granular control over the skill discovery process. Furthermore, a dynamic model monitors progress across these factors, directing the agent’s attention toward areas that have been less explored. This structured methodology not only encourages the emergence of richer, more diverse skills but also produces a factorized skill representation. This representation supports fine-grained, disentangled control over individual entities, thereby facilitating efficient training for compositional downstream tasks through Hierarchical Reinforcement Learning (HRL).

Our experiments, conducted across three environments featuring between one and ten factors, show that SUSD can identify diverse and complex skills without supervision. The results indicate that our method significantly surpasses current unsupervised skill discovery techniques, particularly in complex and factorized settings. The code for this work is publicly accessible at: https://github.com/hadi-hosseini/SUSD.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC