Title: WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis

Abstract:

As Large Language Models (LLMs) evolve, their scope is expanding beyond the creation of static interfaces to the construction of executable, interactive worlds driven by natural language prompts. A logical progression in this domain is browser-native 3D environments, typically developed using Three.js. Generating such systems presents unique challenges: the resulting code must seamlessly integrate assets, adhere to spatial and physical laws, and ensure that user-facing controls remain synchronized with the underlying, hidden runtime state. However, current benchmarks and evaluation methods for web generation largely rely on superficial metrics, such as analyzing pixels or DOM nodes, failing to capture the complex mechanics occurring within the opaque context of a Three.js world.

To address this gap, we introduce WorldCoder-Bench, a comprehensive benchmark designed for the autonomous synthesis of 3D worlds grounded in physical principles. The dataset comprises 2,026 tasks meticulously curated by experts, spanning Simulation, Rendering, and Application scenarios. These tasks include optional .glb assets and enforce hidden behavioral contracts to test system robustness. Additionally, we present StateProbe, a novel execution-based protocol. This tool inspects generated programs within a sandboxed browser environment, verifying hidden contracts that are hardened against mutations, focusing specifically on runtime states and their transitions.

In addition to measuring verification coverage, we introduce new utility metrics: Return on Automation and Time Efficiency Multiplier, which quantify cost-effectiveness and time savings adjusted for correctness. Our evaluation of nine leading frontier models reveals that even the top-performing system achieves a verification coverage of only 27.8% on WorldCoder-Core and 19.9% on WorldCoder-Robust. The primary sources of failure are identified as state-schema drift and disrupted interaction chains, rather than the absence of visual scene elements. Furthermore, our analysis of utility metrics indicates that less expensive or faster models can still deliver significant value when applied to simpler domains. WorldCoder-Bench is publicly accessible at https://anonymous.4open.science/r/WorldCoder-Bench/.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC