When scaling trillion-parameter models like Kimi K3 across Cerebras WSE-3T wafers, which architectural priority is most critical for frontier AI performance?
Scaling Kimi K3: The Cerebras WSE-3T Pipeline Strategy
Deploying frontier-level models like the 3-trillion parameter Kimi K3 presents a massive memory challenge. A single Cerebras WSE-3T wafer possesses 44 GB of on-chip memory, whereas trillion-parameter models require terabytes of space for weights alone. To solve this, Cerebras employs a sophisticated pipeline strategy, splitting the model across multiple WSE-3T wafers-requiring approximately 59 wafers for Kimi K3. In this architecture, each wafer acts like a stage in an assembly line where model weights remain stationary on the chip, and only activations pass between wafers via RDMA over Ethernet. This design allows for incredible speeds, reaching 1000-2000+ tokens per second. However, engineers face a fundamental trade-off: adjusting micro-batch sizes. Utilizing smaller micro-batches optimizes for ultra-fast latency for individual requests, while larger micro-batches are designed to maximize the total throughput of the system. This technological leap shifts the focus from simply managing weight storage to optimizing the flow of data through an interconnected wafer-scale network.
Options
- Maximizing total throughput using larger micro-batches
- Minimizing individual request latency with smaller micro-batches
- Expanding on-chip memory capacity on each WSE-3T wafer
- Optimizing the communication speed between pipeline stages