Title: Simplicial Embeddings Enhance Sample Efficiency in Actor-Critic Agents

Abstract:

While recent studies have sought to accelerate the wall-clock training of actor-critic methods through extensive environment parallelization, these approaches often still demand a substantial volume of environment interactions to reach target performance levels. To address this, we introduce simplicial embeddings—lightweight representation layers that impose simplicial structures on embeddings. Building on the premise that well-structured representations boost both the generalization capabilities and sample efficiency of deep reinforcement learning (RL) agents, our method leverages this geometric inductive bias to generate sparse, discrete features. These features help stabilize critic bootstrapping and reinforce policy gradients. Our experiments demonstrate that integrating simplicial embeddings into FastTD3, FastSAC, and PPO yields consistent gains in both sample efficiency and final performance across various continuous and discrete control tasks, all while maintaining original runtime speeds.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC