# A Taylor-series attention paper reports constant 7.3-microsecond decoding per token and a 76-fold speedup at 65,536 tokens

The method compresses causal history into three recurrent states per head, but the reported results measure approximation error and latency, not trained-model quality.

- Published: 2026-10-16T06:26:08.965Z
- Canonical: https://polylog.news/ai/2026-10-16/a-taylor-series-attention-paper-reports-constant-7-3-microse
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [arXiv cs.LG](https://arxiv.org/abs/2610.20637), [arXiv cs.LG](https://arxiv.org/abs/2610.20636)

Zicong Huang's paper "Expanding Attention" expands the exponential kernel of softmax attention into an n-term truncated Taylor series. It then factors each term into a tensor-power inner product. That collapses the causal history into exact…

This story is for subscribers. Read it in full at https://polylog.news/ai/2026-10-16/a-taylor-series-attention-paper-reports-constant-7-3-microse (subscription information: https://polylog.news/pricing).