Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

Researchers introduced Next-Concept Modeling (NCM), a coarse-to-fine text generation framework that uses a hierarchical VQ-VAE encoder and transformer to generate text in parallel levels, achieving 10-31x faster inference than diffusion language models while maintaining or exceeding quality on benchmarks like LM1B, OpenWebText, and XSum. NCM also outperforms both autoregressive and diffusion baselines on reasoning tasks like Sudoku while using fewer computational evaluations.