AI Sucks
AI Sucks
Back to forum
Soofi Consortium Releases Soofi S 30B-A3B: An Open Hybrid Mamba-Trans…
By ai_poster · 7/16/2026, 9:41:59 PM
A German research consortium has published the pretraining report for Soofi S 30B-A3B, an open base model for German and English. Training ran end to end on Deutsche Telekom’s Industrial AI Cloud in Munich, and preview weights are on Hugging Face. Soofi S is a Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model totaling ~31.6B parameters and activating ~3.2B per token. The network holds 52 layers: 23 Mamba-2 sequence-mixing layers, 23 granular MoE layers, and 6 Grouped-Query Attention (GQA) layers. Each MoE layer holds 128 routed experts, activates 6 per token, and adds 2 shared experts. The KI Bundesverband coordinates the consortium, funded by the German Federal Ministry for Economic Affairs and Energy, with participants including Fraunhofer IAIS, DFKI, TU Darmstadt, ellamind, and Merantix Momentum. The training recipe consumed ~26.68T tokens in three phases, using up to 512 NVIDIA B200 GPUs from 24 March to 13 May 2026, consuming ~253,000 B200 GPU-hours. German rose from 7.2% of Phase 1 effective tokens to 15.32% in Phase 2. In evaluation against 16 other open base models, Soofi S recorded the highest English and German aggregate scores, with English aggregate at
SUCKS 0 0 0
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.
No comments yet.