# Sonnet 5.5's jump from 10.3% to 70.6% on Terminal-Bench 4.0 tests how far vendor agent evals can be trusted

Two new arXiv benchmarks argue for scoring agents in closed loops and on calibrated probabilities. XiangqiBench has agents play full games against a responding opponent. The Turkish-language HakemBench scores a probability for every option.

- Published: 2026-10-05T06:00:43.302Z
- Canonical: https://polylog.news/ai/2026-10-05/sonnet-5-5-s-jump-from-10-3-to-70-6-on-terminal-bench-4-0-te
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [Anthropic News (Claude Sonnet 5.5)](https://www.anthropic.com/claude-sonnet-5-5), [arXiv cs.CL (XiangqiBench)](https://arxiv.org/abs/2610.02425), [arXiv cs.CL (HakemBench)](https://arxiv.org/abs/2610.02293), [Vals AI](https://www.vals.ai/models/anthropic_claude-sonnet-5-5)

Anthropic reports that Claude Sonnet 5.5 scores 70.6% on Terminal-Bench 4.0. According to The Decoder, its predecessor, Sonnet 5, scored 10.3%. That is a gain of about 60 points in one release on a command-line agent benchmark. The new Sonn…

This story is for subscribers. Read it in full at https://polylog.news/ai/2026-10-05/sonnet-5-5-s-jump-from-10-3-to-70-6-on-terminal-bench-4-0-te (subscription information: https://polylog.news/pricing).