Title: 3DThinkVLA: Empowering Vision-Language-Action Models with Latent 3D Priors Through 3D-Thinking-Guided Co-training

Original: arXiv:2606.04436v1 Announce Type: new Abstract: We introduce a co-training framework guided by 3D thinking, which allows vision-language-action (VLA) models to conduct implicit 3D spatial reasoning during the process of action prediction. The fundamental premise of our approach is that 3D geometry perception and 3D spatial reasoning are separate abilities; these can be separated and integrated into various levels of feature hierarchy. Within the latent space, three interconnected components collaborate during training: (1) A latent 3D geometry perception module captures low-level geometric cues by aligning intermediate visual features with a 3D foundation model to obtain geometric priors, all without altering the VLM backbone architecture. (2) An online 3D reasoning distillation module addresses the reasoning gap caused by prompts through a shared reasoning anchor token. In the context of 3D VLM co-training, this anchor is generated as the initial output token to effectively encode spatial priors. During VLA training, this token is positioned as an input between task and action instructions, thereby transferring high-level spatial reasoning from explicit teacher prompts to student action prompts, bypassing the need for chain-of-thought text generation. (3) These separated geometric and reasoning features are merged via spatially augmented action integration, which simultaneously injects them into action-query tokens as hierarchical spatial conditions to avoid action shortcuts. When deployed, our approach keeps only lightweight adapters for implicit 3D reasoning, eliminating the need for the 3D foundation model and the teacher branch used for supervision. As a result, it functions solely on 2D images, requiring no 3D sensors, external models, or explicit text generation, while preventing catastrophic forgetting of the pretrained VLM and achieving state-of-the-art results on LIBERO, LIBERO-PLUS, SimplerEnv, and real-world manipulation tasks.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC