Diffusion Forcing: Next-Token Prediction Meets Full-Sequence Diffusion

Diffusion Forcing is introduced as a novel training paradigm that combines the strengths of full-sequence diffusion models and next-token models for sequence generative modeling. By training a diffusion model to denoise tokens with varying noise levels, Diffusion Forcing can generate future tokens without diffusing past ones, offering benefits such as variable-length generation and guiding sampling to desired trajectories. This approach optimizes a variational lower bound on token likelihoods and allows for flexible behaviors like stabilizing auto-regressive rollout and planning with causal uncertainty. The method excels in video prediction tasks, outperforming baselines in stability and consistency. It enables stable infinite rollout without a sliding window, showcasing its stabilization effect. Additionally, Diffusion Forcing can be used for diffusion planning and long-horizon imitation learning tasks, demonstrating success in non-Markovian scenarios where traditional techniques fail. The method's ability to handle noisy observations enhances robustness in real-world applications.