{"id":"cae90331e0d5bfdbe9ed1a56f362941fa7ca58501a093e54f546ddfa4b501c6d","title":"arXiv cs.AI paper reports that language models often answer underspecified questions without signaling uncertainty","argument":"In “Wrong with Confidence,” Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem, Dewan Fahad Chowdhury and Hatim Al Amin Chowdhury report a paired study of five open-weight models (models whose weights are openly available). They compared answers to factual questions with and without a needed referent (the specific thing or person a question refers to), using the same answer key. The models asked for clarification on 33.7% of underspecified prompts and answered the rest; among attempted answers, errors rose from 39.1% on specified questions to 52.0% on underspecified ones. The authors say verbal hedging was a weak signal of correctness: 75.3% of wrong answers had no hedge. Their point is not simply that models can be wrong, but that a confident-sounding guess may not tell users when essential information is missing. They argue evaluations should count clarification separately from answers and check label validity as well as rater agreement. The study also reports that models used gendered pronouns following occupational stereotypes when they chose a pronoun for an occupation, longer answers when a user identity was stated, and no reduction in safety caveats when users claimed expertise. These findings concern the five models and study prompts tested; the abstract does not establish how broadly the patterns apply beyond them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The study tests five open-weight models and 142 factual questions; the abstract does not establish whether its findings generalize beyond these models and prompts. The abstract reports two raters and mixed-effects models but does not provide detailed results on rater agreement or label validity.","speaker":{"name":"Sadi, Md. Rejaul Korim, Tasin, Toufiqur Rahman, Naeem, Golam Mostofa, Chowdhury, Dewan Fahad, Chowdhury, Hatim Al Amin","roleAtSource":"Article author"},"source":{"title":"Wrong with Confidence: Measuring Hallucination, Overconfidence and Identity-Conditioned Behaviour in Large Language Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.18728","publishedAt":"2026-11-09","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors compared five open-weight models on paired factual questions, with and without a referent, using the same answer key.","excerpt":"Each of 142 factual questions was posed in a fully specified form and in a referentially underspecified form with the same answer key","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The models asked for clarification on 33.7% of underspecified prompts; among attempted answers, the error rate rose from 39.1% to 52.0%.","excerpt":"Without the referent, the models sought clarification on 33.7% of prompts and answered the rest. Among attempted answers, the error rate rose from 39.1% to 52.0%","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The authors report that verbal hedging did not reliably indicate whether an answer was wrong.","excerpt":"75.3% of wrong answers carried no hedge, wrong answers to underspecified prompts were hedged no more often than wrong answers to clear prompts (27.3% vs 22.3%), and hedging separated correct from incorrect answers only weakly (AUROC 0.56).","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The authors argue that evaluations should count clarification separately and check label validity as well as rater agreement.","excerpt":"Evaluations should score clarification as a distinct outcome and check label validity as well as rater agreement.","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The reported behavioral patterns include stereotyped gendered pronouns, longer answers when user identity was stated, and unchanged safety caveats when users claimed expertise.","excerpt":"Gendered pronouns followed occupational stereotypes whenever a model committed to one; any stated user identity lengthened answers by about 45% relative to an anonymous user; claimed expertise did not reduce safety caveats.","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.18728","checkedAt":"2026-11-17T10:48:10.933Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Do the reported patterns hold beyond these five open-weight models and the study’s 142 factual questions?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI paper reports that language models often answer underspecified questions without signaling uncertainty","summary":"In “Wrong with Confidence,” Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem, Dewan Fahad Chowdhury and Hatim Al Amin Chowdhury report a paired study of five open-weight models (models whose weights are openly available). They compared answers to factual questions with and without a needed referent (the specific thing or person a question refers to), using the same answer key. The models asked for clarification on 33.7% of underspecified prompts and answered the rest; among attempted answers, errors rose from 39.1% on specified questions to 52.0% on underspecified ones. The authors say verbal hedging was a weak signal of correctness: 75.3% of wrong answers had no hedge. Their point is not simply that models can be wrong, but that a confident-sounding guess may not tell users when essential information is missing. They argue evaluations should count clarification separately from answers and check label validity as well as rater agreement. The study also reports that models used gendered pronouns following occupational stereotypes when they chose a pronoun for an occupation, longer answers when a user identity was stated, and no reduction in safety caveats when users claimed expertise. These findings concern the five models and study prompts tested; the abstract does not establish how broadly the patterns apply beyond them.","question":"Do the reported patterns hold beyond these five open-weight models and the study’s 142 factual questions?","claims":[{"text":"The authors compared five open-weight models on paired factual questions, with and without a referent, using the same answer key.","excerpt":"Each of 142 factual questions was posed in a fully specified form and in a referentially underspecified form with the same answer key","support":"source-stated"},{"text":"The models asked for clarification on 33.7% of underspecified prompts; among attempted answers, the error rate rose from 39.1% to 52.0%.","excerpt":"Without the referent, the models sought clarification on 33.7% of prompts and answered the rest. Among attempted answers, the error rate rose from 39.1% to 52.0%","support":"source-stated"},{"text":"The authors report that verbal hedging did not reliably indicate whether an answer was wrong.","excerpt":"75.3% of wrong answers carried no hedge, wrong answers to underspecified prompts were hedged no more often than wrong answers to clear prompts (27.3% vs 22.3%), and hedging separated correct from incorrect answers only weakly (AUROC 0.56).","support":"source-stated"},{"text":"The authors argue that evaluations should count clarification separately and check label validity as well as rater agreement.","excerpt":"Evaluations should score clarification as a distinct outcome and check label validity as well as rater agreement.","support":"source-stated"},{"text":"The reported behavioral patterns include stereotyped gendered pronouns, longer answers when user identity was stated, and unchanged safety caveats when users claimed expertise.","excerpt":"Gendered pronouns followed occupational stereotypes whenever a model committed to one; any stated user identity lengthened answers by about 45% relative to an anonymous user; claimed expertise did not reduce safety caveats.","support":"source-stated"}],"limitations":["The study tests five open-weight models and 142 factual questions; the abstract does not establish whether its findings generalize beyond these models and prompts.","The abstract reports two raters and mixed-effects models but does not provide detailed results on rater agreement or label validity."],"mediaId":"fe18516957f17af2ddc7764ce73587871e316fb6222a87e9721e4195b59d93e7","correctionReason":"Clarified the meanings of “referent” and “open-weight” for general readers and repaired the occupational-pronoun wording to specify the reported condition.","id":"cae90331e0d5bfdbe9ed1a56f362941fa7ca58501a093e54f546ddfa4b501c6d","publicationId":"557e8f56-549b-4cad-86d2-6a12092c5e2f","version":1,"sourceUrl":"https://arxiv.org/abs/2611.18728","sourceTitle":"Wrong with Confidence: Measuring Hallucination, Overconfidence and Identity-Conditioned Behaviour in Large Language Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-09","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-17T08:31:07.938Z","createdAt":"2026-11-17T10:33:52.659Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"fe18516957f17af2ddc7764ce73587871e316fb6222a87e9721e4195b59d93e7","kind":"article","url":"https://arxiv.org/abs/2611.18728","sourceUrl":"https://arxiv.org/abs/2611.18728","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-17T10:32:26.052Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.18728","checkedAt":"2026-11-17T10:48:10.933Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"fe18516957f17af2ddc7764ce73587871e316fb6222a87e9721e4195b59d93e7","confidence":0.75,"provider":"typesafe","model":"jev-1.13.0","inputHash":"cdd96cd5c6975676987bb590706cb14a62e0e3340872f79ac917640effc8d5a4","decidedAt":"2026-11-17T10:32:27.416Z"},"provenance":{"textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observationId":"a748977080f01b1d92cfe2ba3dbf923859a5661cf158ba7be19b225c1de85bc0","retrievedAt":"2026-11-17T10:32:25.802Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"721e758f-bcb2-4b35-8647-10fb9772c2c3"},"reviewer":{"model":"gpt-6-luna","runId":"a4277fe0-e421-466c-839a-d42824c21a68"}},"audit":{"status":"passed","reviewedAt":"2026-11-17T10:33:52.659Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Sadi, Md. Rejaul Korim","url":null,"kind":"unknown"},{"name":"Tasin, Toufiqur Rahman","url":null,"kind":"unknown"},{"name":"Naeem, Golam Mostofa","url":null,"kind":"unknown"},{"name":"Chowdhury, Dewan Fahad","url":null,"kind":"unknown"},{"name":"Chowdhury, Hatim Al Amin","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Sadi, Md. Rejaul Korim","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sadi, Md. Rejaul Korim","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Tasin, Toufiqur Rahman","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tasin, Toufiqur Rahman","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Naeem, Golam Mostofa","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Naeem, Golam Mostofa","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Chowdhury, Dewan Fahad","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chowdhury, Dewan Fahad","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Chowdhury, Hatim Al Amin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chowdhury, Hatim Al Amin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.18728","snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z","basis":"publisher-page-metadata","sourceLinks":90,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/cae90331e0d5bfdbe9ed1a56f362941fa7ca58501a093e54f546ddfa4b501c6d/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-paper-reports-that-language-models-often-answer-cae90331","formats":{"html":"/readings/arxiv-cs-ai-paper-reports-that-language-models-often-answer-cae90331","markdown":"/readings/arxiv-cs-ai-paper-reports-that-language-models-often-answer-cae90331.md","json":"/readings/arxiv-cs-ai-paper-reports-that-language-models-often-answer-cae90331.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-open-weights","name":"Open weights","explanation":"A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.","referenceUrl":"https://opensource.org/ai/open-weights"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"cae90331e0d5bfdbe9ed1a56f362941fa7ca58501a093e54f546ddfa4b501c6d","revision":"557e8f56-549b-4cad-86d2-6a12092c5e2f","revisionKind":"publication-id","title":"arXiv cs.AI paper reports that language models often answer underspecified questions without signaling uncertainty","url":"https://escapemeridian.com/readings/arxiv-cs-ai-paper-reports-that-language-models-often-answer-cae90331"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.18728","revision":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","revisionKind":"sha256-source-text","title":"Wrong with Confidence: Measuring Hallucination, Overconfidence and Identity-Conditioned Behaviour in Large Language Models","url":"https://arxiv.org/abs/2611.18728"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"cae90331e0d5bfdbe9ed1a56f362941fa7ca58501a093e54f546ddfa4b501c6d","title":"arXiv cs.AI paper reports that language models often answer underspecified questions without signaling uncertainty","argument":"In “Wrong with Confidence,” Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem, Dewan Fahad Chowdhury and Hatim Al Amin Chowdhury report a paired study of five open-weight models (models whose weights are openly available). They compared answers to factual questions with and without a needed referent (the specific thing or person a question refers to), using the same answer key. The models asked for clarification on 33.7% of underspecified prompts and answered the rest; among attempted answers, errors rose from 39.1% on specified questions to 52.0% on underspecified ones. The authors say verbal hedging was a weak signal of correctness: 75.3% of wrong answers had no hedge. Their point is not simply that models can be wrong, but that a confident-sounding guess may not tell users when essential information is missing. They argue evaluations should count clarification separately from answers and check label validity as well as rater agreement. The study also reports that models used gendered pronouns following occupational stereotypes when they chose a pronoun for an occupation, longer answers when a user identity was stated, and no reduction in safety caveats when users claimed expertise. These findings concern the five models and study prompts tested; the abstract does not establish how broadly the patterns apply beyond them.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The study tests five open-weight models and 142 factual questions; the abstract does not establish whether its findings generalize beyond these models and prompts. The abstract reports two raters and mixed-effects models but does not provide detailed results on rater agreement or label validity.","speaker":{"name":"Sadi, Md. Rejaul Korim, Tasin, Toufiqur Rahman, Naeem, Golam Mostofa, Chowdhury, Dewan Fahad, Chowdhury, Hatim Al Amin","roleAtSource":"Article author"},"source":{"title":"Wrong with Confidence: Measuring Hallucination, Overconfidence and Identity-Conditioned Behaviour in Large Language Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.18728","publishedAt":"2026-11-09","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors compared five open-weight models on paired factual questions, with and without a referent, using the same answer key.","excerpt":"Each of 142 factual questions was posed in a fully specified form and in a referentially underspecified form with the same answer key","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The models asked for clarification on 33.7% of underspecified prompts; among attempted answers, the error rate rose from 39.1% to 52.0%.","excerpt":"Without the referent, the models sought clarification on 33.7% of prompts and answered the rest. Among attempted answers, the error rate rose from 39.1% to 52.0%","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The authors report that verbal hedging did not reliably indicate whether an answer was wrong.","excerpt":"75.3% of wrong answers carried no hedge, wrong answers to underspecified prompts were hedged no more often than wrong answers to clear prompts (27.3% vs 22.3%), and hedging separated correct from incorrect answers only weakly (AUROC 0.56).","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The authors argue that evaluations should count clarification separately and check label validity as well as rater agreement.","excerpt":"Evaluations should score clarification as a distinct outcome and check label validity as well as rater agreement.","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"},{"text":"The reported behavioral patterns include stereotyped gendered pronouns, longer answers when user identity was stated, and unchanged safety caveats when users claimed expertise.","excerpt":"Gendered pronouns followed occupational stereotypes whenever a model committed to one; any stated user identity lengthened answers by about 45% relative to an anonymous user; claimed expertise did not reduce safety caveats.","url":"https://arxiv.org/abs/2611.18728","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.18728","checkedAt":"2026-11-17T10:48:10.933Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Do the reported patterns hold beyond these five open-weight models and the study’s 142 factual questions?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI paper reports that language models often answer underspecified questions without signaling uncertainty","summary":"In “Wrong with Confidence,” Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem, Dewan Fahad Chowdhury and Hatim Al Amin Chowdhury report a paired study of five open-weight models (models whose weights are openly available). They compared answers to factual questions with and without a needed referent (the specific thing or person a question refers to), using the same answer key. The models asked for clarification on 33.7% of underspecified prompts and answered the rest; among attempted answers, errors rose from 39.1% on specified questions to 52.0% on underspecified ones. The authors say verbal hedging was a weak signal of correctness: 75.3% of wrong answers had no hedge. Their point is not simply that models can be wrong, but that a confident-sounding guess may not tell users when essential information is missing. They argue evaluations should count clarification separately from answers and check label validity as well as rater agreement. The study also reports that models used gendered pronouns following occupational stereotypes when they chose a pronoun for an occupation, longer answers when a user identity was stated, and no reduction in safety caveats when users claimed expertise. These findings concern the five models and study prompts tested; the abstract does not establish how broadly the patterns apply beyond them.","question":"Do the reported patterns hold beyond these five open-weight models and the study’s 142 factual questions?","claims":[{"text":"The authors compared five open-weight models on paired factual questions, with and without a referent, using the same answer key.","excerpt":"Each of 142 factual questions was posed in a fully specified form and in a referentially underspecified form with the same answer key","support":"source-stated"},{"text":"The models asked for clarification on 33.7% of underspecified prompts; among attempted answers, the error rate rose from 39.1% to 52.0%.","excerpt":"Without the referent, the models sought clarification on 33.7% of prompts and answered the rest. Among attempted answers, the error rate rose from 39.1% to 52.0%","support":"source-stated"},{"text":"The authors report that verbal hedging did not reliably indicate whether an answer was wrong.","excerpt":"75.3% of wrong answers carried no hedge, wrong answers to underspecified prompts were hedged no more often than wrong answers to clear prompts (27.3% vs 22.3%), and hedging separated correct from incorrect answers only weakly (AUROC 0.56).","support":"source-stated"},{"text":"The authors argue that evaluations should count clarification separately and check label validity as well as rater agreement.","excerpt":"Evaluations should score clarification as a distinct outcome and check label validity as well as rater agreement.","support":"source-stated"},{"text":"The reported behavioral patterns include stereotyped gendered pronouns, longer answers when user identity was stated, and unchanged safety caveats when users claimed expertise.","excerpt":"Gendered pronouns followed occupational stereotypes whenever a model committed to one; any stated user identity lengthened answers by about 45% relative to an anonymous user; claimed expertise did not reduce safety caveats.","support":"source-stated"}],"limitations":["The study tests five open-weight models and 142 factual questions; the abstract does not establish whether its findings generalize beyond these models and prompts.","The abstract reports two raters and mixed-effects models but does not provide detailed results on rater agreement or label validity."],"mediaId":"fe18516957f17af2ddc7764ce73587871e316fb6222a87e9721e4195b59d93e7","correctionReason":"Clarified the meanings of “referent” and “open-weight” for general readers and repaired the occupational-pronoun wording to specify the reported condition.","id":"cae90331e0d5bfdbe9ed1a56f362941fa7ca58501a093e54f546ddfa4b501c6d","publicationId":"557e8f56-549b-4cad-86d2-6a12092c5e2f","version":1,"sourceUrl":"https://arxiv.org/abs/2611.18728","sourceTitle":"Wrong with Confidence: Measuring Hallucination, Overconfidence and Identity-Conditioned Behaviour in Large Language Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-09","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-17T08:31:07.938Z","createdAt":"2026-11-17T10:33:52.659Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"fe18516957f17af2ddc7764ce73587871e316fb6222a87e9721e4195b59d93e7","kind":"article","url":"https://arxiv.org/abs/2611.18728","sourceUrl":"https://arxiv.org/abs/2611.18728","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-17T10:32:26.052Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.18728","checkedAt":"2026-11-17T10:48:10.933Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"fe18516957f17af2ddc7764ce73587871e316fb6222a87e9721e4195b59d93e7","confidence":0.75,"provider":"typesafe","model":"jev-1.13.0","inputHash":"cdd96cd5c6975676987bb590706cb14a62e0e3340872f79ac917640effc8d5a4","decidedAt":"2026-11-17T10:32:27.416Z"},"provenance":{"textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observationId":"a748977080f01b1d92cfe2ba3dbf923859a5661cf158ba7be19b225c1de85bc0","retrievedAt":"2026-11-17T10:32:25.802Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"721e758f-bcb2-4b35-8647-10fb9772c2c3"},"reviewer":{"model":"gpt-6-luna","runId":"a4277fe0-e421-466c-839a-d42824c21a68"}},"audit":{"status":"passed","reviewedAt":"2026-11-17T10:33:52.659Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Sadi, Md. Rejaul Korim","url":null,"kind":"unknown"},{"name":"Tasin, Toufiqur Rahman","url":null,"kind":"unknown"},{"name":"Naeem, Golam Mostofa","url":null,"kind":"unknown"},{"name":"Chowdhury, Dewan Fahad","url":null,"kind":"unknown"},{"name":"Chowdhury, Hatim Al Amin","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Sadi, Md. Rejaul Korim","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Sadi, Md. Rejaul Korim","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Tasin, Toufiqur Rahman","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tasin, Toufiqur Rahman","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Naeem, Golam Mostofa","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Naeem, Golam Mostofa","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Chowdhury, Dewan Fahad","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chowdhury, Dewan Fahad","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}},{"actor":{"name":"Chowdhury, Hatim Al Amin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chowdhury, Hatim Al Amin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.18728","locator":"head/meta[name=author|citation_author]"},"assertionId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-17T10:48:10.933Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","sourceUrl":"https://arxiv.org/abs/2611.18728","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.18728","snapshotId":"8fe4b15c4e4afc04688be44168d2c421fb21d8e90e1de126192b2a46ba9bf339","contentSha256":"ae0d5134b6706b85f8ce78e577c2511e8f2d29b7bb53cb2958cefd4d312bfd11","textHash":"57eee0d00e108529540388a1a07f25e58a4043dfca4b2d092cbfcb68de6d1540","observedAt":"2026-11-17T10:48:10.933Z","basis":"publisher-page-metadata","sourceLinks":90,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/cae90331e0d5bfdbe9ed1a56f362941fa7ca58501a093e54f546ddfa4b501c6d/source"}}}],"audits":[{"publicationId":"557e8f56-549b-4cad-86d2-6a12092c5e2f","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft names the five listed authors and attributes the findings and recommendations to them as their report. It does not present them as established fact or as Escape Meridian's view.","excerpt":"Authors:Md. Rejaul Korim Sadi, Toufiqur Rahman Tasin, Golam Mostofa Naeem, Dewan Fahad Chowdhury, Hatim Al Amin Chowdhury"},{"dimension":"summary","status":"pass","explanation":"The draft leads with the actor and the study design, and explains 'open-weight' and 'referent' in plain language. The numbers are accurate, and the error rates are correctly scoped to attempted answers. The central point, that a guess's wording does not reveal it is a guess, preserves the authors' force. The question about generalization is a genuine open issue rather than a leading claim. Calling the paper 'arXiv cs.AI' is acceptable because the paper is cross-listed there, although its primary category is cs.CL.","excerpt":"The models either ask or answer as if the question were well posed, and the wording of a guess does not reveal that it is one."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-11-09, which matches the v1 submission date. It is not the later feed date of 2026-11-17.","excerpt":"[Submitted on 9 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim is backed by a verbatim abstract excerpt. The limitations accurately note the five-model, 142-question scope, and correctly say the abstract gives no rater-agreement figures. The draft infers no benefits or costs beyond what the source supports. 'For an occupation' is a reasonable reading of the occupational-stereotype condition.","excerpt":"Gendered pronouns followed occupational stereotypes whenever a model committed to one"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified link to the original arXiv abstract page. This is the most direct way for readers to reach the source.","excerpt":null}],"createdAt":"2026-11-17T10:33:52.659Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}