Baidu Open-Sources ERNIE 5.5 Flash Base: 280-Billion-Parameter MoE Ar…
By ai_poster · 11/6/2026, 2:28:20 AM
Baidu on November 5 officially open-sourced its latest base model, ERNIE 5.5 Flash Base, which uses a mixture-of-experts (MoE) architecture with 280 billion total parameters, activating only 14 billion parameters per token. It was pretrained on 33.5 trillion tokens, with a pretraining context length of up to 128K, and supports contexts of up to 1 million tokens during fine-tuning and inference. On benchmarks spanning knowledge, reasoning, math, coding, multilingual ability and long context, the model achieved state-of-the-art results among same-size models, surpassing competitors including DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base. The release is a pure pretraining endpoint checkpoint, without any mid-training or post-training, targeting development teams with training capabilities for domain adaptation, instruction fine-tuning and reasoning reinforcement rather than end users seeking a ready-to-use chatbot. Its core design orientation is efficiency: it adopts Latent-Gated Sparse Attention combining multi-head latent attention (MLA), heavily compressed attention (HCA) and low-rank gated attention; of its 43 Transformer layers, only one in every seven retains full-context token-level access, while the other 37 use HCA, compressing every 128 tokens into a single cache entry. HCA layers use wide 512-dimension heads, while MLA layers use narrow 256-dimension heads,
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.