Cerebras Deploys Trillion-Parameter AI Models

Cerebras has successfully deployed trillion-parameter AI models across multiple wafer-scale chips by splitting models across numerous WSE-3T wafers, each containing 44 GB of on-chip memory. Using the 3-trillion-parameter Kimi K3 model as an example, the company demonstrates that it requires 59 wafers total to run the model, achieving inference speeds of 1000-2000+ tokens per second by strategically placing dense attention layers (one per wafer) and linear attention layers (two per wafer) based on their memory requirements. The wafers are connected via RDMA over Ethernet with approximately 2-microsecond latency between hops, enabling efficient token flow across the distributed system.