Morning Edition · Monday, August 31, 2026Published at 2:23 AM EDT · New York

Interpretability Researchers Put Superposition on a Formal Footing Using Compressed Sensing

A new paper models feature superposition as sparse recovery through an overcomplete dictionary, giving the field conditions under which a network's features can be read out at all.

Mechanistic interpretability has run for years on an empirical observation: neural networks pack more features into a layer than it has dimensions, representing them as overlapping directions rather than dedicated neurons. That observation,…