Morning Edition · Tuesday, July 21, 2026Published at 1:32 AM EDT · New York

A New Router Aims to Make Mixture-of-Experts Selection More Consistent

The method conditions expert routing on multi-level context rather than shallow per-token representations, targeting a known instability in sparse models.

A new arXiv paper targets a weakness in how mixture-of-experts (MoE) models route tokens. MoE scales transformers efficiently by sending each token to a small subset of experts, but the authors note that existing routers typically decide ba…