Title: RoboTrustBench: Assessing the Reliability of Video World Models in Robotic Manipulation Tasks

Video world models are gaining traction in the field of robotic manipulation; however, current evaluation methods predominantly test these systems using instructions that are valid, feasible, and safe. To address this gap, we present RoboTrustBench, a novel benchmark designed to measure the trustworthiness of video world models across four distinct scenarios: Normal, Constraint-Sensitive, Counterfactual, and Adversarial.

Derived from real-world DROID episodes, the benchmark comprises 1,207 instruction-image pairs that have been validated by experts. It employs a comprehensive evaluation framework consisting of a six-dimensional protocol and 13 specific, fine-grained criteria. In our study, we assessed seven prominent video world models using both human reviewers and Multimodal Large Language Model (MLLM) evaluations.

Our findings reveal a disparity in model performance: while existing models frequently produce visually coherent videos, they encounter significant difficulties in areas such as constraint reasoning, counterfactual grounding, physical interaction simulation, and the suppression of unsafe instructions. These outcomes indicate that achieving high visual fidelity and superficial adherence to instructions is not enough to ensure trustworthiness in robotic video world modeling.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC