Title: FocusDiT: Enhancing Fine-Grained Image Synthesis via Query Masking in Diffusion Transformers

Original: arXiv:2606.02090v1 Announce Type: new Abstract: Diffusion transformer (DiT) has been widely adopted in the generative diffusion field, advancing the denoising of query tokens through attention and Feed-Forward (\text{FFN}) layers. FFN actually acts as the key-value vocabulary for decoding visual contents where the value embeds the visual semantical knowledge. We present that focusing on critical query tokens corresponding to more complex details and encouraging the model to improve these tokens is essential for fine-grained visual generation. To this end, we propose FocusDiT, which applies a Masking scheme to focus on critical query tokens that are exclusively fed into FFN. The masked queries can retrieve visual tokens from the FFN vocabularies, and use them to decode their visual details. Extensive text-to-image experiments validate the effectiveness of token masking in enhancing generative performance.

Rewritten: Title: FocusDiT: Leveraging Query Masking in Diffusion Transformers for High-Resolution Image Synthesis

Original: arXiv:2606.02090v1 Announce Type: new Abstract: The Diffusion Transformer (DiT) architecture has gained significant traction within generative diffusion models, primarily by refining query tokens via attention mechanisms and Feed-Forward Network (\text{FFN}) layers. In this context, the FFN functions as a key-value repository for interpreting visual data, with its values encoding semantic visual information. Our research highlights that prioritizing specific query tokens associated with intricate details and actively promoting their refinement is crucial for achieving fine-grained visual outputs. Consequently, we introduce FocusDiT, a method that employs a masking strategy to isolate these vital query tokens, directing them exclusively toward the FFN. By masking these queries, the model can access visual tokens stored in the FFN’s vocabulary to reconstruct detailed visual elements. Comprehensive text-to-image evaluations confirm that this token masking approach significantly boosts generative quality.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC