Title: World-Task Factorization for Robot Learning

Abstract

For robot learning to yield policies capable of generalizing across novel combinations of constraints, collaborators, and environments, it is essential to structurally decompose the policy architecture. This structural choice fundamentally determines which aspects of behavior generalize, which necessitate retraining, and which remain conflated. Current approaches vary significantly, ranging from relying on data scaling to allow structure to emerge spontaneously, to manually engineering it through hierarchies, skill libraries, or learned specializations. In this study, we examine what we propose as the most foundational factorization in robotics: the separation of the world from the task. We explore the conditions that render this separation theoretically sound. World factors consist of properties inherent to the embodied system and its environment, existing independently of any specific intent. Conversely, task factors are determined by the logical requirements of the task relative to the possibilities afforded by the world. We formalize this asymmetry using Bayesian model evidence, demonstrating that this approach aligns with the data-generating process, sustains high likelihood via an analytical world model, and minimizes the Occam razor penalty on task parameters. To implement this factorization, we combine AICON—a differentiable graph composed of recursive estimators and interconnections that is compositional, requires no task-specific data, and propagates cost gradients to actuators—with a compact, learned policy that modulates these gradient paths. Gradients act as the interface between these two factors, transmitting world structure through the graph and task structure via costs, thereby facilitating low-dimensional learning while maintaining structural generalization capabilities. We evaluate this world/task factorization across three diverse problems involving heterogeneous robots, environments, task logics, and sensorimotor modalities. Our framework surpasses both end-to-end baselines and analytical heuristics in all tested scenarios, achieving zero-shot generalization to out-of-distribution configurations and seamless transfer to real hardware without the need for retraining.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC