|
Baidu released ERNIE-5.5-Flash-Base, a sparse Mixture-of-Experts language model with 280B total parameters (14B activated per token) pre-trained on 33.5T tokens with support for up to 1M-token context. The base model weights are offered without fine-tuning to serve as an efficient starting point for developers to build domain-specific applications, instruction tuning, and reinforcement learning research. The architecture prioritizes efficiency across all components, including sparse attention and MoE layers, designed to reduce compute and memory costs for further training and inference.
Read Full Article →
← Back to latest