Morning Edition · Monday, October 12, 2026Published at 2:16 AM EDT · New York

Small-model papers test linear attention, integer number formats and new sequence models for compute-limited training

Kairos pairs bidirectional gated DeltaNet with sliding-window attention for a byte-level multimodal model trained on one machine, and a second paper proposes algebraic integer formats that reduce inference to integer arithmetic.

"Kairos: Toward a Byte-Level Foundation Model for Every Modality" starts from a constraint most labs do not face. A small language model trained on a single local machine leaves little budget for experimenting with architectures. Kairos com…