# StepFun Releases Step 5 Preview, a 600-Billion-Parameter Sparse Model With Weights Due October 15

Only 27 billion parameters activate per token, and the company says a modified attention scheme enables the one-million-token context window.

- Published: 2026-09-21T06:21:17.793Z
- Canonical: https://polylog.news/ai/2026-09-21/stepfun-releases-step-5-preview-a-600-billion-parameter-spar
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [Polylog editors](https://polylog.news), [Pandaily](https://pandaily.com/stepfun-step-5-preview-600b-moe-1m-context), [Hugging Face](https://huggingface.co/SHSLab/Step-5-Preview-BF16)

The Chinese lab StepFun has announced Step 5 Preview, a sparse mixture-of-experts (MoE) model with 600 billion total parameters of which roughly 27 billion activate for any given token. The company positions it for long-running agent tasks, software development and complex analysis, and says the application programming interface (API) is live now with downloadable weights scheduled for October 15, [according to coverage of the launch](https://pandaily.com/stepfun-step-5-preview-600b-moe-1m-context). The Russian-language channel AI ML Big Data [carried the same specification](https://t.me/ai_machinelearning_big_data/10972) hours after the announcement.

The interesting engineering claim is not the parameter count but how the one-million-token context is served. StepFun describes a sparse variant of grouped-query attention with block-wise token merging, which uses an index to pull only the historical blocks relevant to the current step into the attention computation rather than attending densely across the full prompt. That is the same problem several papers posted to arXiv this weekend attack from the academic side, and it is the real constraint on million-token context: prefill cost and key-value cache bandwidth, not the ability to store the tokens.

On evals, restraint is warranted. StepFun's own table places Step 5 Preview level with Kimi K3 and GLM 5.3 on most rows and behind GPT-6 Astra and Claude Opus 5 on the hardest coding and agentic rows, which is a vendor's self-assessment and reads as one. Third-party scoring so far is thin. Weights are already mirrored on Hugging Face ahead of the official October date, so independent reproduction on public benchmarks should arrive quickly rather than remaining a matter of trust.

The activation ratio is the number that carries commercial weight. Serving 27 billion active parameters out of 600 billion means the marginal cost per token looks closer to a mid-sized dense model than to a frontier one, provided the operator can hold the full weight set in memory. That is the standard argument for sparse architectures, and it is also the reason a 600-billion-parameter open-weight release matters differently than it would have two years ago: the hardware floor for running it is set by memory capacity, which is falling in price more slowly than compute.

## What this means

A downloadable 600-billion-parameter model with a million-token window creates competitive pressure on the mid-tier of the closed API market rather than on the frontier itself, because the customers most likely to switch are the ones buying long-context document and codebase processing, where price per million tokens matters more than differences in raw capability. StepFun gains distribution it could not simply purchase. Western API vendors selling long-context tiers lose pricing power on that specific workload. The gating factor for adoption is memory capacity on the serving side, which routes demand toward accelerators rich in high-bandwidth memory and toward hosted inference providers rather than toward on-premises deployment.

## What to watch

- Whether the weights actually publish on October 15 under a permissive licence, since a research-only licence would confine the release to evaluation and blunt the competitive effect.
- Independent scores on agentic and coding benchmarks run by third parties, which will show whether the sparse attention scheme holds accuracy at long context or degrades once the index misses relevant blocks.
- Whether Western inference providers list the model, which is the clearest signal of whether enterprise buyers will run Chinese open weights in production or only in testing.