{"id":"a1a291dd64bc087cb5075a27b8f7588775b49ec025e8ff00b4298822d33c8db9","title":"arXiv cs.AI: AI Isn't Getting Dumber, We're Getting Pickier","argument":"Authors Zihan Chen, Lei Nico Zheng and Di Zhu argue that complaints about AI assistants getting worse may reflect changing human standards, not just changes to the models. They examine blind votes on Chatbot Arena, where people compare answers from two anonymous large language models (LLMs), using pairs whose model versions were fixed. For the same unchanged pairs, votes saying “both are bad” increased, while the stronger model’s share of decisive votes stayed flat. The authors report that the pattern holds within the same voter and model pair, across language and model types, and with prompt and interface changes accounted for. They interpret this split—worsening absolute verdicts but stable relative comparisons—as an “evaluation drift”: as the category improves, people may judge a product that stands still more harshly. That distinction matters, they argue, for reading LLM leaderboards and for human feedback used to train models. The evidence comes from 1,260,967 votes on Chatbot Arena between May 2023 and August 2024; the paper describes itself as a working paper.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence described here is from Chatbot Arena votes cast between May 2023 and August 2024, not other evaluation settings. The study compares pairs whose model versions were fixed; the abstract does not establish how the findings apply to changing model versions. The source identifies the paper as a working paper.","speaker":{"name":"Chen, Zihan, Zheng, Lei Nico, Zhu, Di","roleAtSource":"Article author"},"source":{"title":"AI Isn't Getting Dumber, We're Getting Pickier: Rising Standards in Human Evaluation of Large Language Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.14333","publishedAt":"2026-11-10","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors study blind human votes on Chatbot Arena for pairs of models whose versions were fixed.","excerpt":"We study 1,260,967 blind votes cast from May 2023 to August 2024 on pairs of models whose versions are fixed","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"For unchanged model pairs, the share of votes saying “both are bad” rose over the study period.","excerpt":"For the same pair of unchanged models, the share of votes saying \"both are bad\" rises by 1.71 percentage points a month","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"The stronger model’s share of decisive votes did not trend in 2024, even as “both are bad” votes rose.","excerpt":"The ranking does not trend: in 2024, the stronger model's share of decisive votes is flat.","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"The authors interpret the pattern as evaluation drift: standards may rise with progress in the category, making a stationary product seem to decline.","excerpt":"The pattern fits a standard that follows the category's progress, so a product that stands still is judged as declining.","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"The authors say this evaluation drift matters for LLM leaderboards and human feedback used to train models.","excerpt":"an evaluation drift that matters for LLM leaderboards and for the human feedback used to train models.","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14333","checkedAt":"2026-11-12T03:54:10.863Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would the rise in “both are bad” votes also appear in human evaluations beyond the Chatbot Arena votes studied here?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: AI Isn't Getting Dumber, We're Getting Pickier","summary":"Authors Zihan Chen, Lei Nico Zheng and Di Zhu argue that complaints about AI assistants getting worse may reflect changing human standards, not just changes to the models. They examine blind votes on Chatbot Arena, where people compare answers from two anonymous large language models (LLMs), using pairs whose model versions were fixed. For the same unchanged pairs, votes saying “both are bad” increased, while the stronger model’s share of decisive votes stayed flat. The authors report that the pattern holds within the same voter and model pair, across language and model types, and with prompt and interface changes accounted for. They interpret this split—worsening absolute verdicts but stable relative comparisons—as an “evaluation drift”: as the category improves, people may judge a product that stands still more harshly. That distinction matters, they argue, for reading LLM leaderboards and for human feedback used to train models. The evidence comes from 1,260,967 votes on Chatbot Arena between May 2023 and August 2024; the paper describes itself as a working paper.","question":"Would the rise in “both are bad” votes also appear in human evaluations beyond the Chatbot Arena votes studied here?","claims":[{"text":"The authors study blind human votes on Chatbot Arena for pairs of models whose versions were fixed.","excerpt":"We study 1,260,967 blind votes cast from May 2023 to August 2024 on pairs of models whose versions are fixed","support":"source-stated"},{"text":"For unchanged model pairs, the share of votes saying “both are bad” rose over the study period.","excerpt":"For the same pair of unchanged models, the share of votes saying \"both are bad\" rises by 1.71 percentage points a month","support":"source-stated"},{"text":"The stronger model’s share of decisive votes did not trend in 2024, even as “both are bad” votes rose.","excerpt":"The ranking does not trend: in 2024, the stronger model's share of decisive votes is flat.","support":"source-stated"},{"text":"The authors interpret the pattern as evaluation drift: standards may rise with progress in the category, making a stationary product seem to decline.","excerpt":"The pattern fits a standard that follows the category's progress, so a product that stands still is judged as declining.","support":"source-stated"},{"text":"The authors say this evaluation drift matters for LLM leaderboards and human feedback used to train models.","excerpt":"an evaluation drift that matters for LLM leaderboards and for the human feedback used to train models.","support":"source-stated"}],"limitations":["The evidence described here is from Chatbot Arena votes cast between May 2023 and August 2024, not other evaluation settings.","The study compares pairs whose model versions were fixed; the abstract does not establish how the findings apply to changing model versions.","The source identifies the paper as a working paper."],"mediaId":"cea0a3d01c999e2bb1b98af134c46055847a099dfb1becb9c95c7f302951f87e","correctionReason":null,"id":"a1a291dd64bc087cb5075a27b8f7588775b49ec025e8ff00b4298822d33c8db9","publicationId":"0190af6e-0964-4778-8a1c-8eb0cefd21a9","version":1,"sourceUrl":"https://arxiv.org/abs/2611.14333","sourceTitle":"AI Isn't Getting Dumber, We're Getting Pickier: Rising Standards in Human Evaluation of Large Language Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-10","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-11T05:06:35.111Z","createdAt":"2026-11-12T01:47:48.213Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"cea0a3d01c999e2bb1b98af134c46055847a099dfb1becb9c95c7f302951f87e","kind":"article","url":"https://arxiv.org/abs/2611.14333","sourceUrl":"https://arxiv.org/abs/2611.14333","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T01:47:02.256Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14333","checkedAt":"2026-11-12T03:54:10.863Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v6","mediaId":"cea0a3d01c999e2bb1b98af134c46055847a099dfb1becb9c95c7f302951f87e","confidence":0.63,"provider":"typesafe","model":"jev-1.13.0","inputHash":"1ae3f1447d7a4e725a930f982099aa4006d45d14b4d969572aa4d8894a94c3a6","decidedAt":"2026-11-12T01:47:02.953Z"},"provenance":{"textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observationId":"630a1fff1adf714461c930d34a644fa6ee07990a5c0fbd8afd644b1764eaeb99","retrievedAt":"2026-11-12T01:47:02.164Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"b47fa79e-66ed-49dc-8469-d82e4166d016"},"reviewer":{"model":"gpt-6-luna","runId":"c2736fb6-2791-43dc-899f-1ed7b7a839af"}},"audit":{"status":"passed","reviewedAt":"2026-11-12T01:47:48.213Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Chen, Zihan","url":null,"kind":"unknown"},{"name":"Zheng, Lei Nico","url":null,"kind":"unknown"},{"name":"Zhu, Di","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Chen, Zihan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Zihan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14333","locator":"head/meta[name=author|citation_author]"},"assertionId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T03:54:10.863Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","sourceUrl":"https://arxiv.org/abs/2611.14333","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z"}},{"actor":{"name":"Zheng, Lei Nico","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zheng, Lei Nico","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14333","locator":"head/meta[name=author|citation_author]"},"assertionId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T03:54:10.863Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","sourceUrl":"https://arxiv.org/abs/2611.14333","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z"}},{"actor":{"name":"Zhu, Di","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhu, Di","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14333","locator":"head/meta[name=author|citation_author]"},"assertionId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T03:54:10.863Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","sourceUrl":"https://arxiv.org/abs/2611.14333","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.14333","snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/a1a291dd64bc087cb5075a27b8f7588775b49ec025e8ff00b4298822d33c8db9/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-ai-isn-t-getting-dumber-we-re-getting-pickier-a1a291dd","formats":{"html":"/readings/arxiv-cs-ai-ai-isn-t-getting-dumber-we-re-getting-pickier-a1a291dd","markdown":"/readings/arxiv-cs-ai-ai-isn-t-getting-dumber-we-re-getting-pickier-a1a291dd.md","json":"/readings/arxiv-cs-ai-ai-isn-t-getting-dumber-we-re-getting-pickier-a1a291dd.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"a1a291dd64bc087cb5075a27b8f7588775b49ec025e8ff00b4298822d33c8db9","revision":"0190af6e-0964-4778-8a1c-8eb0cefd21a9","revisionKind":"publication-id","title":"arXiv cs.AI: AI Isn't Getting Dumber, We're Getting Pickier","url":"https://escapemeridian.com/readings/arxiv-cs-ai-ai-isn-t-getting-dumber-we-re-getting-pickier-a1a291dd"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.14333","revision":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","revisionKind":"sha256-source-text","title":"AI Isn't Getting Dumber, We're Getting Pickier: Rising Standards in Human Evaluation of Large Language Models","url":"https://arxiv.org/abs/2611.14333"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"a1a291dd64bc087cb5075a27b8f7588775b49ec025e8ff00b4298822d33c8db9","title":"arXiv cs.AI: AI Isn't Getting Dumber, We're Getting Pickier","argument":"Authors Zihan Chen, Lei Nico Zheng and Di Zhu argue that complaints about AI assistants getting worse may reflect changing human standards, not just changes to the models. They examine blind votes on Chatbot Arena, where people compare answers from two anonymous large language models (LLMs), using pairs whose model versions were fixed. For the same unchanged pairs, votes saying “both are bad” increased, while the stronger model’s share of decisive votes stayed flat. The authors report that the pattern holds within the same voter and model pair, across language and model types, and with prompt and interface changes accounted for. They interpret this split—worsening absolute verdicts but stable relative comparisons—as an “evaluation drift”: as the category improves, people may judge a product that stands still more harshly. That distinction matters, they argue, for reading LLM leaderboards and for human feedback used to train models. The evidence comes from 1,260,967 votes on Chatbot Arena between May 2023 and August 2024; the paper describes itself as a working paper.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence described here is from Chatbot Arena votes cast between May 2023 and August 2024, not other evaluation settings. The study compares pairs whose model versions were fixed; the abstract does not establish how the findings apply to changing model versions. The source identifies the paper as a working paper.","speaker":{"name":"Chen, Zihan, Zheng, Lei Nico, Zhu, Di","roleAtSource":"Article author"},"source":{"title":"AI Isn't Getting Dumber, We're Getting Pickier: Rising Standards in Human Evaluation of Large Language Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.14333","publishedAt":"2026-11-10","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors study blind human votes on Chatbot Arena for pairs of models whose versions were fixed.","excerpt":"We study 1,260,967 blind votes cast from May 2023 to August 2024 on pairs of models whose versions are fixed","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"For unchanged model pairs, the share of votes saying “both are bad” rose over the study period.","excerpt":"For the same pair of unchanged models, the share of votes saying \"both are bad\" rises by 1.71 percentage points a month","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"The stronger model’s share of decisive votes did not trend in 2024, even as “both are bad” votes rose.","excerpt":"The ranking does not trend: in 2024, the stronger model's share of decisive votes is flat.","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"The authors interpret the pattern as evaluation drift: standards may rise with progress in the category, making a stationary product seem to decline.","excerpt":"The pattern fits a standard that follows the category's progress, so a product that stands still is judged as declining.","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"},{"text":"The authors say this evaluation drift matters for LLM leaderboards and human feedback used to train models.","excerpt":"an evaluation drift that matters for LLM leaderboards and for the human feedback used to train models.","url":"https://arxiv.org/abs/2611.14333","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14333","checkedAt":"2026-11-12T03:54:10.863Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would the rise in “both are bad” votes also appear in human evaluations beyond the Chatbot Arena votes studied here?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: AI Isn't Getting Dumber, We're Getting Pickier","summary":"Authors Zihan Chen, Lei Nico Zheng and Di Zhu argue that complaints about AI assistants getting worse may reflect changing human standards, not just changes to the models. They examine blind votes on Chatbot Arena, where people compare answers from two anonymous large language models (LLMs), using pairs whose model versions were fixed. For the same unchanged pairs, votes saying “both are bad” increased, while the stronger model’s share of decisive votes stayed flat. The authors report that the pattern holds within the same voter and model pair, across language and model types, and with prompt and interface changes accounted for. They interpret this split—worsening absolute verdicts but stable relative comparisons—as an “evaluation drift”: as the category improves, people may judge a product that stands still more harshly. That distinction matters, they argue, for reading LLM leaderboards and for human feedback used to train models. The evidence comes from 1,260,967 votes on Chatbot Arena between May 2023 and August 2024; the paper describes itself as a working paper.","question":"Would the rise in “both are bad” votes also appear in human evaluations beyond the Chatbot Arena votes studied here?","claims":[{"text":"The authors study blind human votes on Chatbot Arena for pairs of models whose versions were fixed.","excerpt":"We study 1,260,967 blind votes cast from May 2023 to August 2024 on pairs of models whose versions are fixed","support":"source-stated"},{"text":"For unchanged model pairs, the share of votes saying “both are bad” rose over the study period.","excerpt":"For the same pair of unchanged models, the share of votes saying \"both are bad\" rises by 1.71 percentage points a month","support":"source-stated"},{"text":"The stronger model’s share of decisive votes did not trend in 2024, even as “both are bad” votes rose.","excerpt":"The ranking does not trend: in 2024, the stronger model's share of decisive votes is flat.","support":"source-stated"},{"text":"The authors interpret the pattern as evaluation drift: standards may rise with progress in the category, making a stationary product seem to decline.","excerpt":"The pattern fits a standard that follows the category's progress, so a product that stands still is judged as declining.","support":"source-stated"},{"text":"The authors say this evaluation drift matters for LLM leaderboards and human feedback used to train models.","excerpt":"an evaluation drift that matters for LLM leaderboards and for the human feedback used to train models.","support":"source-stated"}],"limitations":["The evidence described here is from Chatbot Arena votes cast between May 2023 and August 2024, not other evaluation settings.","The study compares pairs whose model versions were fixed; the abstract does not establish how the findings apply to changing model versions.","The source identifies the paper as a working paper."],"mediaId":"cea0a3d01c999e2bb1b98af134c46055847a099dfb1becb9c95c7f302951f87e","correctionReason":null,"id":"a1a291dd64bc087cb5075a27b8f7588775b49ec025e8ff00b4298822d33c8db9","publicationId":"0190af6e-0964-4778-8a1c-8eb0cefd21a9","version":1,"sourceUrl":"https://arxiv.org/abs/2611.14333","sourceTitle":"AI Isn't Getting Dumber, We're Getting Pickier: Rising Standards in Human Evaluation of Large Language Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-10","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-11T05:06:35.111Z","createdAt":"2026-11-12T01:47:48.213Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"cea0a3d01c999e2bb1b98af134c46055847a099dfb1becb9c95c7f302951f87e","kind":"article","url":"https://arxiv.org/abs/2611.14333","sourceUrl":"https://arxiv.org/abs/2611.14333","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T01:47:02.256Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14333","checkedAt":"2026-11-12T03:54:10.863Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v6","mediaId":"cea0a3d01c999e2bb1b98af134c46055847a099dfb1becb9c95c7f302951f87e","confidence":0.63,"provider":"typesafe","model":"jev-1.13.0","inputHash":"1ae3f1447d7a4e725a930f982099aa4006d45d14b4d969572aa4d8894a94c3a6","decidedAt":"2026-11-12T01:47:02.953Z"},"provenance":{"textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observationId":"630a1fff1adf714461c930d34a644fa6ee07990a5c0fbd8afd644b1764eaeb99","retrievedAt":"2026-11-12T01:47:02.164Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"b47fa79e-66ed-49dc-8469-d82e4166d016"},"reviewer":{"model":"gpt-6-luna","runId":"c2736fb6-2791-43dc-899f-1ed7b7a839af"}},"audit":{"status":"passed","reviewedAt":"2026-11-12T01:47:48.213Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Chen, Zihan","url":null,"kind":"unknown"},{"name":"Zheng, Lei Nico","url":null,"kind":"unknown"},{"name":"Zhu, Di","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Chen, Zihan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chen, Zihan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14333","locator":"head/meta[name=author|citation_author]"},"assertionId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T03:54:10.863Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","sourceUrl":"https://arxiv.org/abs/2611.14333","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z"}},{"actor":{"name":"Zheng, Lei Nico","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zheng, Lei Nico","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14333","locator":"head/meta[name=author|citation_author]"},"assertionId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T03:54:10.863Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","sourceUrl":"https://arxiv.org/abs/2611.14333","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z"}},{"actor":{"name":"Zhu, Di","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhu, Di","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14333","locator":"head/meta[name=author|citation_author]"},"assertionId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T03:54:10.863Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","sourceUrl":"https://arxiv.org/abs/2611.14333","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.14333","snapshotId":"3a6086ad6f93d81d9100c77692b3af669e99560919de4cc0ee9193feaf557972","contentSha256":"0719fcbd25b2189a8535638e6a48304a728edb8971e463e65e251846a4e42306","textHash":"e3f02d3074e141ded1d8f32717f8975ecdee864b49ae60489e89860e0bb2877b","observedAt":"2026-11-12T03:54:10.863Z","basis":"publisher-page-metadata","sourceLinks":44,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/a1a291dd64bc087cb5075a27b8f7588775b49ec025e8ff00b4298822d33c8db9/source"}}}],"audits":[{"publicationId":"0190af6e-0964-4778-8a1c-8eb0cefd21a9","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading correctly credits Zihan Chen, Lei Nico Zheng and Di Zhu with the paper on arXiv. It keeps the findings and the evaluation-drift interpretation attributed to the authors, using phrases like 'the authors report' and 'they argue'.","excerpt":"Authors:Zihan Chen, Lei Nico Zheng, Di Zhu"},{"dimension":"summary","status":"pass","explanation":"The reading leads with the authors' thesis and explains Chatbot Arena and LLMs in plain terms. It conveys the central split between rising absolute complaints and stable relative rankings, and keeps the hedged 'may' framing. The term 'decisive votes' is understandable in context. The title and question are faithful, and the question raises a genuine unresolved issue about generalization.","excerpt":"Absolute verdicts on AI drifted while relative comparisons held steady"},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-10, which matches the arXiv v1 submission date. The later feed entry is dated 2026-11-11, but the earlier submission date is the correct original publication date.","excerpt":"[Submitted on 10 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim is directly supported by the abstract. The 'flat' ranking claim correctly keeps the 2024 scope. Robustness claims about voters, pairs, languages, model types, prompt controls and interface periods are stated in the source. The significance for leaderboards and training feedback is attributed to the authors. The limitations are accurate, and no unsupported benefits or costs are added.","excerpt":"It holds with prompt controls, in every major language, for dated vendor snapshots, open-weight models and models with stable answer length, and between interface changes."},{"dimension":"media","status":"pass","explanation":"The selected media item is the verified direct link to the arXiv abstract page for this paper. It is the most direct way for a reader to access the source.","excerpt":null}],"createdAt":"2026-11-12T01:47:48.213Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}