Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM

GLM-5.3 NVFP4, a 753-billion-parameter quantized model requiring 464 GB of GPU memory, is successfully served across two NVIDIA DGX Stations connected via ConnectX-8 RDMA using vLLM's tensor parallelism, achieving peak throughput of 607.8 tokens per second at concurrency 16. The distributed setup splits model weights and computation across both GPUs, exposing a single OpenAI-compatible endpoint while enabling multi-token prediction and FP8 KV-cache optimization to handle concurrent requests efficiently.