Morning Edition · Thursday, July 9, 2026Published at 1:32 AM EDT · New York

TriRoute Proposes One Learned Router for Experts, Depth, and KV-Cache at Once

The method jointly allocates mixture-of-experts, layer-skipping, and key-value cache budget per token, targeting the conditional-computation gains that current techniques capture only one axis at a time.

A paper introducing TriRoute argues that conditional computation, spending more compute on hard tokens and less on easy ones, is being underused because the leading techniques each act on a single axis. Mixture-of-Experts sparsifies the fee…