Morning Edition · Monday, October 5, 2026Published at 2:00 AM EDT · New York

Sonnet 5.5's jump from 10.3% to 70.6% on Terminal-Bench 4.0 tests how far vendor agent evals can be trusted

Two new arXiv benchmarks argue for scoring agents in closed loops and on calibrated probabilities. XiangqiBench has agents play full games against a responding opponent. The Turkish-language HakemBench scores a probability for every option.

Anthropic reports that Claude Sonnet 5.5 scores 70.6% on Terminal-Bench 4.0. According to The Decoder, its predecessor, Sonnet 5, scored 10.3%. That is a gain of about 60 points in one release on a command-line agent benchmark. The new Sonn…