Smaller, faster, safer: running Kimi and GLM at scale

Cloudflare uses three optimization techniques—KV cache quantization, model weight compression, and shared cache protection—to efficiently run large language models like Kimi K-series and GLM on its Workers AI platform. By storing the KV cache in 8-bit precision instead of 16-bit, the company can double the context window capacity and achieve 41% higher throughput at lower cost, though individual tokens process slightly slower. These methods allow Cloudflare to serve more customers simultaneously on the same hardware without sacrificing model accuracy.