# Ai2's OLMo Hybrid Argues Attention Plus Linear RNNs Beats Either Alone

The Allen Institute's 7-billion-parameter hybrid interleaves transformer attention with Gated DeltaNet, reaching OLMo 3's accuracy on the Massive Multitask Language Understanding (MMLU) benchmark using 49 percent fewer tokens.

- Published: 2026-06-27T10:45:08.127Z
- Canonical: https://polylog.news/ai/2026-06-27/ai2-s-olmo-hybrid-argues-attention-plus-linear-rnns-beats-ei
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [Polylog editors](https://polylog.news)

The Allen Institute for AI published the work behind OLMo Hybrid, a 7-billion-parameter model that interleaves standard transformer attention layers with Gated DeltaNet, a modern linear recurrent neural network. The Russian-language summary…

This story is for subscribers. Read it in full at https://polylog.news/ai/2026-06-27/ai2-s-olmo-hybrid-argues-attention-plus-linear-rnns-beats-ei (subscription information: https://polylog.news/pricing).