Morning Edition · Friday, August 14, 2026Published at 2:27 AM EDT · New York

Three New Preprints Attack the Same Cost: Attention That Grows With Context

Papers posted Thursday propose locality-aware attention with separated knowledge memory, content-routed recurrent state, and recurrent depth retrofitted into an already-trained model.

Three preprints published the same day take three different approaches to the cost that dominates both training and serving for large language models: attention that scales quadratically with sequence length during training, and leaves a ke…