Morning Edition · Monday, September 21, 2026Published at 2:21 AM EDT · New York

Three New Papers Attack the Prefill and Key-Value Cache Costs That Make Long Context Expensive

Radius-bounded sparse prefill, learned per-context sparsity during decoding, and attention-informed expert routing all target the same cost constraint from different points in the stack.

A cluster of papers posted to arXiv on Monday converges on the same economic problem: long-context inference is now dominated by costs that have nothing to do with how many parameters a model has. RBS-Attention targets prefill, the phase wh…