# Changing only the gold labels moved a multilingual personal-data benchmark score by up to 7.55 F1 points

With each system's output held fixed, switching the annotation set moved one score by 4.95 F1 points and the other by 2.00. F1 is a combined measure of precision and recall. A companion paper asks what similarity scores measure when the structure they claim to detect is absent.

- Published: 2026-10-06T06:03:25.241Z
- Canonical: https://polylog.news/ai/2026-10-06/changing-only-the-gold-labels-moved-a-multilingual-personal
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [arXiv cs.CL: Same Output, Different Gold](https://arxiv.org/abs/2610.03825), [arXiv cs.CL: The Score Is Not the Structure](https://arxiv.org/abs/2610.03827)

A benchmark score compares a system's output with a reference. Nearly all methodological scrutiny goes to the output. Same Output, Different Gold, by Parth Kulshreshtha, Shivali Dalmia and Abhishek Mukherji, measures the reference side. The…

This story is for subscribers. Read it in full at https://polylog.news/ai/2026-10-06/changing-only-the-gold-labels-moved-a-multilingual-personal (subscription information: https://polylog.news/pricing).