Continuous Diffusion Language Models (CDLM's)

Recent research activity shows continuous diffusion language models are experiencing a resurgence after years of decline, challenging the dominance of discrete diffusion and autoregressive approaches. Early discrete diffusion models emerged in 2021 as an alternative to autoregressive generation, offering theoretical advantages for tasks like infilling and constrained generation, though autoregressive models have since become overwhelmingly dominant. Current renewed interest in continuous diffusion suggests this generative approach may be regaining viability as a research direction.