Morning Edition · Thursday, November 19, 2026Published at 1:16 AM EST · New York
Paired with the Ember optimizer, the design cuts table-optimizer memory by 99.2% relative to AdamW, according to the authors.

N-gram embedding tables add model capacity cheaply. They are large tables of learned vectors for short token sequences that are looked up and added to the input or the hidden states, so they add parameters without adding much compute. Exist…
Track frontier labs, chips, export controls, model releases, regulation, and AI infrastructure.
Track all trends and read every AI and Crypto article for $19/month.
Part of a tracked trend
Frontier Model Efficiency Gains
Capability per unit of training and inference compute keeps improving, letting newer models match prior frontier performance far more cheaply and gradually loosening the link between raw scale and capability.
Start a discussion in Townsquare.
More from this edition
Comments
0No comments yet.