Title: dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching

Abstract:

While Autoregressive Models (ARMs) have historically held the primary position in the Large Language Model ecosystem, diffusion-based Large Language Models (dLLMs) have recently emerged as a promising new paradigm. By iteratively denoising masked segments to generate text, dLLMs demonstrate considerable potential and distinct advantages. Nevertheless, their practical application is often hindered by high inference latency. Standard acceleration strategies used for ARMs, such as Key-Value caching, are ineffective for dLLMs because of the latter's reliance on bidirectional attention mechanisms.

To overcome this bottleneck, our research highlights a critical insight: dLLM inference consists of a fixed prompt and a response that is only partially dynamic. During this process, the majority of tokens exhibit stability across neighboring denoising steps. Leveraging this observation, we introduce dLLM-Cache, a training-free adaptive caching framework. This system integrates long-interval prompt caching with partial response updates that are driven by feature similarity. Consequently, it allows for the efficient recycling of intermediate computations without degrading the model's output quality.

Our extensive evaluations on prominent dLLMs, such as LLaDA 8B and Dream 7B, reveal that dLLM-Cache can reduce FLOPs by as much as 9.1x on the LongBench-HotpotQA benchmark, all while preserving competitive performance. In many scenarios, our approach reduces dLLM inference latency to levels comparable to those of ARMs. The source code for this project is publicly accessible at: https://github.com/maomaocun/dLLM-cache.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC