Title: SimSD: A Streamlined Approach to Speculative Decoding in Diffusion Language Models

Abstract

Diffusion large language models (dLLMs) have recently gained traction as a viable alternative to autoregressive (AR) LLMs, primarily due to their capacity for accelerated inference via parallel or blockwise decoding. Nevertheless, the masked language modeling framework inherent to dLLMs presents a fundamental incompatibility with standard token-level speculative decoding—a technique widely recognized for its efficiency in boosting AR model performance.

In autoregressive decoding, causal masking ensures that token-level contexts remain temporally consistent, thereby allowing a target model to validate multiple proposed tokens within a single forward pass. Conversely, dLLMs utilize bidirectional attention and mask tokens, which causes the effective context to fluctuate across denoising steps. This dynamic nature hinders direct token-level speculative verification.

To address this limitation, we introduce SimSD, a straightforward yet potent speculative decoding algorithm tailored for diffusion language models. The core of SimSD is a plug-and-play masking strategy designed to provide dLLMs with temporally stable, token-level contexts suitable for speculative decoding. Our approach incorporates reference tokens derived from draft-model predictions and employs a specialized attention mask to control their interaction with tokens from the current step. This mechanism enables dLLMs to generate valid logits for drafted tokens in one forward pass, effectively restoring the verification capabilities typically afforded by causal masking in AR models, without sacrificing the parallel decoding benefits of dLLMs.

SimSD is training-free and offers flexible integration with other acceleration methods, including KV cache usage and blockwise decoding. Empirical evaluations on SDAR-family dLLMs across four distinct benchmarks demonstrate that our method can boost decoding throughput by as much as 7.46x, all while maintaining, and in some cases enhancing, the average quality of generated text.

Source: arXiv Generated at: 2026-06-02 00:00:00 UTC