ByteDance is reportedly preparing to pre-train an AI model with roughly 10 trillion parameters, a scale that would surpass every known Chinese AI system and compete with advanced Western labs. The effort would require approximately 30,000 GPUs and an estimated 3 to 6 months of continuous pre-training. This size is more than three times that of Moonshot AI’s Kimi K3, which has 2.8 trillion parameters. The model uses a Mixture of Experts (MoE) architecture, routing inputs to specialized sub-networks for efficiency. ByteDance’s Seed AI team, reportedly around 2,000 staff members, is leading the effort, having previously trained models up to 175 billion parameters using approximately 12,000 GPUs. The company has published research on its MegaScale infrastructure for distributed training systems exceeding 10,000 GPUs. After pre-training, the model would undergo fine-tuning before any potential public release. Founder Zhang Yiming has directed the team to focus on genuine technological breakthroughs rather than distillation from Western AI systems. ByteDance operates TikTok, serving over a billion users globally, and Doubao, generating interaction data for training. US export restrictions on advanced AI chips remain a challenge, but ByteDance’s ability to marshal 30,000 GPUs suggests a workable path. A 10 trillion parameter model would represent a significant leap beyond competitors like Alibaba’s Qwen team, Baidu, DeepSe