Title: TRON: Targeted Rule-Verifiable Online Environments for Visual Reasoning RL

Abstract:

To achieve scalable, verifiable, and controllable training signals for reinforcement learning (RL) in visual reasoning, this study presents TRON (Targeted, Rule-verifiable Online eNvironments). Unlike existing visual RL post-training methods that rely on static, curated datasets with fixed image-question-answer samples constrained by collection limits, TRON operates as an online environment substrate. It utilizes a controllable generator-verifier program to produce training rollouts on demand. This process involves sampling a new latent visual state, rendering an image, posing a question, and precisely verifying the response. Consequently, a single execution can generate an unlimited stream of novel instances tailored to the difficulty requirements of the current curriculum.

The TRON suite currently comprises 520 environments categorized into five distinct ability buckets: spatial, mathematical, diagram, pattern/logic, and counting. This unified substrate facilitates the training of both a single comprehensive model across all buckets and specialized per-bucket ability models, eliminating the need for additional data collection. Furthermore, we provide a comprehensive analysis of the substrate, examining generation reliability, diversity in instances and levels, near-duplicates across environments, and base-model pass rates relative to difficulty. Applying this method to RL post-training yields consistent performance improvements on ten external multimodal reasoning benchmarks, as demonstrated across Qwen3-VL-4B, Qwen2.5-VL-7B, and MiMo-VL-7B-SFT.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC