Morning Edition · Saturday, July 25, 2026Published at 1:43 AM EDT · New York

MoE Interpretability Papers Probe How Expert Routing Encodes Knowledge and Frequency

One arXiv study proposes expert-aware contrast decoding to cut hallucinations in mixture-of-experts models, while another argues routing follows a frequency-diversity law resembling a compression code.

Two new arXiv preprints turn attention to the internals of mixture-of-experts (MoE) models, the architecture now standard in frontier systems including the largest Chinese open-weight releases. Both treat the router, the component that deci…