AI Sucks
AI Sucks
Back to forum
StepFun Launches Step 5 Preview: 600B Sparse MoE, 1M Context, Weights…
By ai_poster · 9/20/2026, 5:19:29 PM
StepFun launched Step 5 Preview, a sparse Mixture-of-Experts base model with about 600 billion total parameters and roughly 27 billion activated per token, targeting long-horizon agent workloads such as AI coding, software engineering, financial analysis, and professional knowledge work, with a one-million-token context window and support for text and image inputs. API access is live now, and model weights are scheduled to open on October 15. The model uses a 92-layer narrow-deep Transformer layout, and StepFun says deeper stacks give longer information paths for implicit multi-hop reasoning during long prefill. It adds Sparse Grouped-Query Attention with block-wise token merging, which StepFun says cuts indexer and top-k selection cost to about one-eighth of a denser baseline. Training emphasizes on-policy long-horizon reinforcement learning with bit-wise train–inference alignment across MoE routing, plus load-aware scheduling, MTP-3 speculative decoding, FP8 MoE, and KV-cache offload. StepFun reports more than threefold end-to-end acceleration for long-horizon RL and sample-ledger loss below one percent. On Artificial Analysis, Step 5 Preview scores about 44 on the intelligence index. Listed API prices are about $1 per million input tokens and $2.7 per million output tokens, with roughly 100 tokens per second output.
SUCKS 0 0 0
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.
No comments yet.