Title: Multimodal Action Diffusion for Robust End-to-End Autonomous Driving

Original: arXiv:2606.02105v1 Announce Type: new

Abstract: Current End-to-End Autonomous Driving (E2E-AD) frameworks predominantly rely on forecasting intermediate trajectory points, leaving the final control execution to rule-based controllers that utilize GPS data. In contrast, the direct prediction of control signals—such as throttle, steering, and braking—within a fully end-to-end pipeline remains largely unexamined, particularly regarding how action multimodality influences such systems. This work posits that abandoning deterministic, single-action outputs is not just an architectural preference but a fundamental factor enhancing driving performance, the quality of learned representations, and training stability. To test this hypothesis, we present the Action Diffusion Transformer (ADT). This anchor-free diffusion transformer utilizes a Mean Squared Error (MSE) loss to naturally capture the multimodal distribution of feasible driving maneuvers. Instead of committing to one fixed command, ADT produces K potential action candidates, choosing the optimal one during inference through Nearest Neighbour Matching (NNM). Our results demonstrate that incorporating action multimodality improves both behavioral consistency and the quality of internal representations—advantages that deterministic models fail to achieve. Furthermore, ADT sets a new state-of-the-art on the rigorous closed-loop Bench2Drive benchmark, accomplishing this with a tenfold reduction in latency. These findings underscore that expressive, multimodal action modeling is not only computationally efficient but also crucial for building robust end-to-end autonomous driving systems.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC