# Three New Preprints Attack the Same Cost: Attention That Grows With Context

Papers posted Thursday propose locality-aware attention with separated knowledge memory, content-routed recurrent state, and recurrent depth retrofitted into an already-trained model.

- Published: 2026-08-14T06:27:18.545Z
- Canonical: https://polylog.news/ai/2026-08-14/three-new-preprints-attack-the-same-cost-attention-that-grow
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [arXiv cs.LG](https://arxiv.org/abs/2608.12419), [arXiv cs.LG](https://arxiv.org/abs/2608.12435), [arXiv cs.CL](https://arxiv.org/abs/2608.11233)

Three preprints published the same day take three different approaches to the cost that dominates both training and serving for large language models: attention that scales quadratically with sequence length during training, and leaves a ke…

This story is for subscribers. Read it in full at https://polylog.news/ai/2026-08-14/three-new-preprints-attack-the-same-cost-attention-that-grow (subscription information: https://polylog.news/pricing).