Title: Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models

Abstract:

Diffusion large language models (dLLMs) provide the benefits of parallel generation and bidirectional attention, allowing them to leverage global context effectively. This capability makes them particularly well-suited for format-constrained tasks, such as generating parseable JSON or adhering to specific reasoning templates. Although simple fixed anchors can be used to enforce these structural requirements, they frequently create rigid spans that result in truncated reasoning processes or unnecessary redundancy. To address these limitations, we introduce Dynamic Infilling Anchors (DIA), a training-free approach that dynamically predicts end-anchor positions to calibrate the length of the generated content prior to iterative infilling. This adaptable strategy maintains both structural integrity and semantic coherence while circumventing the inefficiencies associated with fixed-span techniques. Our experiments on reasoning benchmarks reveal that DIA significantly enhances answer accuracy and format compliance, delivering notable zero-shot performance improvements on the GSM8K and MATH datasets. These findings position DIA as a dependable solution for achieving reliable, structure-aware generation.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC