Morning Edition · Tuesday, July 28, 2026Published at 1:47 AM EDT · New York

CausalGate Prunes Transformer Modules by Causal Importance Rather Than Correlation Heuristics

The paper argues that dropping redundant modules based on hidden-state similarity or activation magnitude misjudges which parts of a model actually matter for a given input.

A paper introducing CausalGate challenges how adaptive-inference methods decide which transformer modules to skip. Existing approaches drop modules using observational heuristics such as hidden-state similarity or activation magnitude, but…