Title: Overcoming Dual Constraints: Transforming Unified Multimodal Models into Self-Adaptive Interleaved Visual Reasoners

Abstract:

Current unified models succeed in merging multimodal comprehension and creation within a singular architecture. Nevertheless, a persistent "understanding-generation gap" remains; while these systems can accurately discern user intent, they frequently struggle to convert that semantic insight into precise, pixel-level control. This disconnect creates two primary hurdles in anything-to-image (X2I) tasks: the attention entanglement bottleneck, which hampers performance on intricate prompts when relying on blind planning, and the visual refinement bottleneck, where unstructured feedback proves inefficient at correcting flaws.

To address these challenges, this study introduces a novel framework that enables unified models to autonomously toggle between different generation strategies, contingent upon both instruction complexity and model capacity. We establish a hierarchical data pipeline that orchestrates execution paths across three distinct adaptive modes: direct generation for straightforward cases, self-reflection for quality enhancement, and multi-step planning to break down complex scenarios.

Leveraging this pipeline, we introduce a premium dataset comprising more than 50,000 samples and employ a two-stage training approach involving Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Our methodology features step-wise reasoning rewards to maintain logical coherence and an intra-group complexity penalty designed to mitigate unnecessary computational costs. Comprehensive experiments confirm that our approach surpasses current baselines in X2I tasks, delivering exceptional generation fidelity across instructions ranging from simple to highly complex. The source code is available at https://github.com/WeChatCV/Interleaved_Visual_Reasoner.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC