StepFun Releases Step 5 Preview, a 600-Billion-Parameter Sparse Model With Weights Due October 15

The Chinese lab StepFun has announced Step 5 Preview, a sparse mixture-of-experts (MoE) model with 600 billion total parameters of which roughly 27 billion activate for any given token. The company positions it for long-running agent tasks, software development and complex analysis, and says the application programming interface (API) is live now with downloadable weights scheduled for October 15, according to coverage of the launch. The Russian-language channel AI ML Big Data carried the same specification hours after the announcement.

The interesting engineering claim is not the parameter count but how the one-million-token context is served. StepFun describes a sparse variant of grouped-query attention with block-wise token merging, which uses an index to pull only the historical blocks relevant to the current step into the attention computation rather than attending densely across the full prompt. That is the same problem several papers posted to arXiv this weekend attack from the academic side, and it is the real constraint on million-token context: prefill cost and key-value cache bandwidth, not the ability to store the tokens.

On evals, restraint is warranted. StepFun's own table places Step 5 Preview level with Kimi K3 and GLM 5.3 on most rows and behind GPT-6 Astra and Claude Opus 5 on the hardest coding and agentic rows, which is a vendor's self-assessment and reads as one. Third-party scoring so far is thin. Weights are already mirrored on Hugging Face ahead of the official October date, so independent reproduction on public benchmarks should arrive quickly rather than remaining a matter of trust.

The activation ratio is the number that carries commercial weight. Serving 27 billion active parameters out of 600 billion means the marginal cost per token looks closer to a mid-sized dense model than to a frontier one, provided the operator can hold the full weight set in memory. That is the standard argument for sparse architectures, and it is also the reason a 600-billion-parameter open-weight release matters differently than it would have two years ago: the hardware floor for running it is set by memory capacity, which is falling in price more slowly than compute.