Morning Edition · Thursday, June 18, 2026Published at 6:48 AM EDT · New York

Gaussian Mixture Attention Proposes Linear-Time Sequence Mixing for Long Contexts

A new arXiv paper replaces dense token-to-token attention with probabilistic latent routing to cut the quadratic cost.

A paper posted to arXiv introduces Gaussian Mixture Attention (GMA), a mechanism that aims to replace the dense token-to-token interaction of standard dot-product attention with a probabilistic latent routing scheme that scales linearly wit…