Morning Edition · Friday, October 16, 2026Published at 2:26 AM EDT · New York

A Taylor-series attention paper reports constant 7.3-microsecond decoding per token and a 76-fold speedup at 65,536 tokens

The method compresses causal history into three recurrent states per head, but the reported results measure approximation error and latency, not trained-model quality.

Zicong Huang's paper "Expanding Attention" expands the exponential kernel of softmax attention into an n-term truncated Taylor series. It then factors each term into a tensor-power inner product. That collapses the causal history into exact…