Title: Video-Mirai: The Case for Foresight in Autoregressive Video Diffusion Models

Abstract:

While causal video generation models rely on historical data to predict forward, they should not restrict their learning exclusively to the past. In the context of streaming autoregressive video diffusion, every generated segment acts as a fixed commitment, requiring subsequent segments to maintain continuity. However, conventional training methods typically focus each causal state solely on explaining the immediate present. This approach fosters a "representation-level planning gap," where states optimized for current segments may inadvertently discard critical information regarding identity, layout, and motion that is essential for future consistency.

To address this, we present Video-Mirai, a training-only intervention that bridges this gap without altering the causal inference process. The method operates by having the generator produce a causal rollout, which is then analyzed by a frozen foresight encoder. This encoder processes the completed rollout non-causally, allowing a lightweight predictor to distill the resulting stopped-gradient targets into the causal states. Crucially, future frames serve to supervise these representations rather than the generator's direct inputs. At the inference stage, both the encoder and predictor are removed, ensuring that the original architecture, per-step FLOPs, and KV-cache behavior remain intact.

Video-Mirai demonstrates significant performance gains over a strong Causal-Forcing baseline. On the 5-second VBench benchmark, the Total Score increased from 83.8 to 84.6. Furthermore, in 30-second rollouts that extend beyond the training horizon, subject consistency rose from 84.9 to 88.5, and background consistency improved from 90.2 to 91.9. Ablation studies confirm that future-conditioned targets are the primary driver of these improvements, while probing reveals that current features become more decodable of future frames. Our findings suggest that while causality should constrain inference, it need not limit representation supervision, highlighting the necessity of foresight in visual autoregressive models.

Project page: https://y0uroy.github.io/Video-Mirai.

Source: arXiv Generated at: 2026-06-03 00:00:00 UTC