Speculative Decoding in vLLM on AMD GPUs

Speculative decoding in vLLM uses a draft-and-verify mechanism to improve LLM serving efficiency on AMD GPUs by having a lightweight draft component propose multiple candidate tokens that a target model then verifies in a single pass, potentially committing multiple tokens at once instead of one token per model step. The technique's effectiveness varies significantly depending on the drafting method, proposal length, model family, and workload characteristics. The article examines five different speculative-drafting approaches (native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark) and provides experimental measurements from AMD Instinct MI300X and MI355X GPUs.