Title: Neural Fields as World Models

Abstract:

Humans frequently simulate potential future scenarios in their minds, such as during mental rehearsal or dreaming, implying that world models can facilitate learning tasks independently of direct environmental interaction. While conventional machine learning world models typically compress visual data into latent vectors, thereby stripping away the spatial organization inherent in the sensory cortex, we introduce isomorphic world models. These architectures maintain sensory topology, transforming physics prediction into geometric propagation instead of abstract state transitions. Our implementation utilizes motor-gated neural fields, in which activity dynamics are driven by local lateral connectivity, while motor commands multiplicatively modulate specific channels. Through three distinct experiments, we demonstrate that this unified architecture can learn ballistic prediction without relying on "teleportation," enhance a catching policy offline by backpropagating task error through a frozen, pre-learned world model, and autonomously develop body-selective motor channels without explicit body labels. Collectively, these findings offer initial evidence that physical prediction, offline task learning, and body-linked representations rely on a shared computational foundation: action-conditional prediction mapped onto spatial structures.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC