Bringing Up DeepSeek-V4-Flash on AMD MI300X

Doubleword documents their efforts to run DeepSeek-V4-Flash on AMD's MI300X accelerator, detailing significant software compatibility challenges that prevent the model from working with vLLM as of May 2026. The primary issues stem from MI300X's use of an outdated FP8 dialect (fnuz) that differs from the newer OCP standard adopted by AMD's later chips, causing numerical precision errors, and missing optimized attention kernels for DeepSeek's sparse attention architecture. Despite MI300X's attractive hardware specifications—192GB HBM3 memory and half the price of NVIDIA's H100—software gaps have prevented it from being a practical option for inference workloads.