Morning Edition · Wednesday, July 15, 2026Published at 1:45 AM EDT · New York

A New Linear-Attention Variant Adds Phase-Controlled Delta Memory to Fix Long-Context State Tracking

Semidirect Fourier Delta Attention keeps a fixed recurrent state instead of a growing key-value cache, using constructive chunk-WY kernels to preserve exact state tracking that linear attention normally loses.

A new paper, Semidirect Fourier Delta Attention, targets the central weakness of linear attention. Linear attention replaces softmax attention's growing key-value cache with a fixed-size recurrent state, which bounds memory and cost, but th…