{"id":"72c7fe47ecbbff7625e259e939fede5fc6cf4a1cf84b23ce41c47af33a13c836","title":"arXiv cs.AI: Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","argument":"Dongsun Moon argues that a high abstention rate does not show that an open language model follows a stated scoring rule. Under that rule, a model should answer only when its probability of being correct exceeds a threshold, t. Tests covered 12 checkpoints from three post-training stages of four fully open 3–8B recipes. At t = 0.75, an abstention format raised mean abstention from 20% to 91%. But within that format, changing t from 0.5 to 0.9 changed abstention by only -0.8 percentage points. Without the format, the higher threshold sometimes lowered abstention; some decisions were inconsistent with any compliant policy using threshold-independent beliefs. In a forced-choice probe, the checkpoints barely responded to t. Moon reports that comparisons with self-assessment signals such as P(True) depended on the grader and answer drift between prompts. An external threshold on recalibrated P(True) made coverage non-increasing by construction. Its results differed across evaluation settings: it beat the abstention format when calibrated post hoc on the evaluation distribution, yet fell below it when calibrated on TriviaQA and applied to PopQA at 0.75. Moon concludes that evaluations should report abstention rates, responsiveness to the stated threshold, and which items models answer.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the paper’s abstract, not its full methods or results; it does not provide details needed to assess the evaluation procedures. The reported tests cover the specified recipes, checkpoints, and probes; the abstract does not establish how the findings extend beyond them.","speaker":{"name":"Moon, Dongsun","roleAtSource":"Article author"},"source":{"title":"Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.14661","publishedAt":"2026-11-10","dateBasis":"Date basis not provided."},"evidence":[{"text":"Moon argues that a high abstention rate alone does not establish compliance with a stated scoring rule.","excerpt":"A high abstention rate is not evidence of compliance","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"At t = 0.75, the abstention format raised mean abstention from 20% to 91%, but changing t from 0.5 to 0.9 within that format changed it by only -0.8 percentage points.","excerpt":"At t = 0.75, a line limiting the reply to an answer or \"I abstain\" (the abstention format) raised the mean abstention rate from 20% to 91%. Yet within this format, raising t from 0.5 to 0.9 changed it by only -0.8 percentage points.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"Without the abstention-format line, raising the threshold even lowered abstention, and up to 40% of a checkpoint’s items received decisions incompatible with a compliant policy using threshold-independent beliefs.","excerpt":"Without the line, the higher threshold even lowered abstention, and up to 40% of a checkpoint's items received decisions that no compliant policy with threshold-independent beliefs could make.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"The checkpoints barely responded to the threshold in a forced-choice probe.","excerpt":"Given the probability that their answer was correct, the checkpoints barely responded to the threshold in a forced-choice probe.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"Results for an external threshold on recalibrated P(True) varied by calibration and evaluation setting.","excerpt":"Calibrated on the evaluation distribution (post hoc), it beat the abstention format at t = 0.5; calibrated on TriviaQA and applied to PopQA, however, it fell below that format at 0.75.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"Moon recommends reporting abstention rates, responsiveness to the stated threshold, and which items models answer.","excerpt":"evaluations using stated scoring rules should report how often models abstain, whether decisions respond to the stated threshold, and which items they answer.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14661","checkedAt":"2026-11-13T11:48:07.139Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How often do the tested models’ decisions respond to the stated threshold, and which items do they answer?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","summary":"Dongsun Moon argues that a high abstention rate does not show that an open language model follows a stated scoring rule. Under that rule, a model should answer only when its probability of being correct exceeds a threshold, t. Tests covered 12 checkpoints from three post-training stages of four fully open 3–8B recipes. At t = 0.75, an abstention format raised mean abstention from 20% to 91%. But within that format, changing t from 0.5 to 0.9 changed abstention by only -0.8 percentage points. Without the format, the higher threshold sometimes lowered abstention; some decisions were inconsistent with any compliant policy using threshold-independent beliefs. In a forced-choice probe, the checkpoints barely responded to t. Moon reports that comparisons with self-assessment signals such as P(True) depended on the grader and answer drift between prompts. An external threshold on recalibrated P(True) made coverage non-increasing by construction. Its results differed across evaluation settings: it beat the abstention format when calibrated post hoc on the evaluation distribution, yet fell below it when calibrated on TriviaQA and applied to PopQA at 0.75. Moon concludes that evaluations should report abstention rates, responsiveness to the stated threshold, and which items models answer.","question":"How often do the tested models’ decisions respond to the stated threshold, and which items do they answer?","claims":[{"text":"Moon argues that a high abstention rate alone does not establish compliance with a stated scoring rule.","excerpt":"A high abstention rate is not evidence of compliance","support":"source-stated"},{"text":"At t = 0.75, the abstention format raised mean abstention from 20% to 91%, but changing t from 0.5 to 0.9 within that format changed it by only -0.8 percentage points.","excerpt":"At t = 0.75, a line limiting the reply to an answer or \"I abstain\" (the abstention format) raised the mean abstention rate from 20% to 91%. Yet within this format, raising t from 0.5 to 0.9 changed it by only -0.8 percentage points.","support":"source-stated"},{"text":"Without the abstention-format line, raising the threshold even lowered abstention, and up to 40% of a checkpoint’s items received decisions incompatible with a compliant policy using threshold-independent beliefs.","excerpt":"Without the line, the higher threshold even lowered abstention, and up to 40% of a checkpoint's items received decisions that no compliant policy with threshold-independent beliefs could make.","support":"source-stated"},{"text":"The checkpoints barely responded to the threshold in a forced-choice probe.","excerpt":"Given the probability that their answer was correct, the checkpoints barely responded to the threshold in a forced-choice probe.","support":"source-stated"},{"text":"Results for an external threshold on recalibrated P(True) varied by calibration and evaluation setting.","excerpt":"Calibrated on the evaluation distribution (post hoc), it beat the abstention format at t = 0.5; calibrated on TriviaQA and applied to PopQA, however, it fell below that format at 0.75.","support":"source-stated"},{"text":"Moon recommends reporting abstention rates, responsiveness to the stated threshold, and which items models answer.","excerpt":"evaluations using stated scoring rules should report how often models abstain, whether decisions respond to the stated threshold, and which items they answer.","support":"source-stated"}],"limitations":["The supplied evidence is the paper’s abstract, not its full methods or results; it does not provide details needed to assess the evaluation procedures.","The reported tests cover the specified recipes, checkpoints, and probes; the abstract does not establish how the findings extend beyond them."],"mediaId":"381c12c20a76be4a3b0844df847e93e3609c813ccf37975cdcbb630e52b57007","correctionReason":null,"id":"72c7fe47ecbbff7625e259e939fede5fc6cf4a1cf84b23ce41c47af33a13c836","publicationId":"560a4d1a-cc97-4219-8792-77d67ca73a43","version":1,"sourceUrl":"https://arxiv.org/abs/2611.14661","sourceTitle":"Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-10","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-13T07:43:10.226Z","createdAt":"2026-11-13T11:17:39.177Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"381c12c20a76be4a3b0844df847e93e3609c813ccf37975cdcbb630e52b57007","kind":"article","url":"https://arxiv.org/abs/2611.14661","sourceUrl":"https://arxiv.org/abs/2611.14661","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-13T11:16:48.955Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14661","checkedAt":"2026-11-13T11:48:07.139Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v7","mediaId":"381c12c20a76be4a3b0844df847e93e3609c813ccf37975cdcbb630e52b57007","confidence":0.6,"provider":"typesafe","model":"jev-1.13.0","inputHash":"a009658c13f8d8ba8a2ac79eda856e39f77d8233b1848726bb23acab6a70680a","decidedAt":"2026-11-13T11:16:49.377Z"},"provenance":{"textHash":"16d991426c50f9cf38cd6944b74f711656729a00ee6c4aa880ff53e0c22d3b59","observationId":"abbe6aee1ddd3541118ca2ebc7599a0036545c22d3473c1cd2ad368422ad7582","retrievedAt":"2026-11-13T11:16:48.906Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d9a03e73-16ee-44cf-8c86-69d356581123"},"reviewer":{"model":"gpt-6-luna","runId":"bae02d66-40e3-4b57-81cd-ecdc124f621b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-13T11:17:39.177Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Moon, Dongsun","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Moon, Dongsun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Moon, Dongsun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14661","locator":"head/meta[name=author|citation_author]"},"assertionId":"d0acc9a68de62e04cf7aaebe3669786a603149715f887a26c76a76f37be55511:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T11:48:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"d0acc9a68de62e04cf7aaebe3669786a603149715f887a26c76a76f37be55511","sourceUrl":"https://arxiv.org/abs/2611.14661","contentSha256":"5187e1390ee93c5fa5a7bfae6e7ba4619bb1329e62fe5720fc1dd7a590ac72a9","textHash":"16d991426c50f9cf38cd6944b74f711656729a00ee6c4aa880ff53e0c22d3b59","observedAt":"2026-11-13T11:48:07.139Z"}}],"participationCoverage":{"total":1,"returned":1,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.14661","snapshotId":"d0acc9a68de62e04cf7aaebe3669786a603149715f887a26c76a76f37be55511","contentSha256":"5187e1390ee93c5fa5a7bfae6e7ba4619bb1329e62fe5720fc1dd7a590ac72a9","textHash":"16d991426c50f9cf38cd6944b74f711656729a00ee6c4aa880ff53e0c22d3b59","observedAt":"2026-11-13T11:48:07.139Z","basis":"publisher-page-metadata","sourceLinks":40,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/72c7fe47ecbbff7625e259e939fede5fc6cf4a1cf84b23ce41c47af33a13c836/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-abstention-is-not-threshold-compliance-stated-72c7fe47","formats":{"html":"/readings/arxiv-cs-ai-abstention-is-not-threshold-compliance-stated-72c7fe47","markdown":"/readings/arxiv-cs-ai-abstention-is-not-threshold-compliance-stated-72c7fe47.md","json":"/readings/arxiv-cs-ai-abstention-is-not-threshold-compliance-stated-72c7fe47.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"72c7fe47ecbbff7625e259e939fede5fc6cf4a1cf84b23ce41c47af33a13c836","revision":"560a4d1a-cc97-4219-8792-77d67ca73a43","revisionKind":"publication-id","title":"arXiv cs.AI: Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","url":"https://escapemeridian.com/readings/arxiv-cs-ai-abstention-is-not-threshold-compliance-stated-72c7fe47"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.14661","revision":"16d991426c50f9cf38cd6944b74f711656729a00ee6c4aa880ff53e0c22d3b59","revisionKind":"sha256-source-text","title":"Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","url":"https://arxiv.org/abs/2611.14661"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"72c7fe47ecbbff7625e259e939fede5fc6cf4a1cf84b23ce41c47af33a13c836","title":"arXiv cs.AI: Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","argument":"Dongsun Moon argues that a high abstention rate does not show that an open language model follows a stated scoring rule. Under that rule, a model should answer only when its probability of being correct exceeds a threshold, t. Tests covered 12 checkpoints from three post-training stages of four fully open 3–8B recipes. At t = 0.75, an abstention format raised mean abstention from 20% to 91%. But within that format, changing t from 0.5 to 0.9 changed abstention by only -0.8 percentage points. Without the format, the higher threshold sometimes lowered abstention; some decisions were inconsistent with any compliant policy using threshold-independent beliefs. In a forced-choice probe, the checkpoints barely responded to t. Moon reports that comparisons with self-assessment signals such as P(True) depended on the grader and answer drift between prompts. An external threshold on recalibrated P(True) made coverage non-increasing by construction. Its results differed across evaluation settings: it beat the abstention format when calibrated post hoc on the evaluation distribution, yet fell below it when calibrated on TriviaQA and applied to PopQA at 0.75. Moon concludes that evaluations should report abstention rates, responsiveness to the stated threshold, and which items models answer.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the paper’s abstract, not its full methods or results; it does not provide details needed to assess the evaluation procedures. The reported tests cover the specified recipes, checkpoints, and probes; the abstract does not establish how the findings extend beyond them.","speaker":{"name":"Moon, Dongsun","roleAtSource":"Article author"},"source":{"title":"Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.14661","publishedAt":"2026-11-10","dateBasis":"Date basis not provided."},"evidence":[{"text":"Moon argues that a high abstention rate alone does not establish compliance with a stated scoring rule.","excerpt":"A high abstention rate is not evidence of compliance","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"At t = 0.75, the abstention format raised mean abstention from 20% to 91%, but changing t from 0.5 to 0.9 within that format changed it by only -0.8 percentage points.","excerpt":"At t = 0.75, a line limiting the reply to an answer or \"I abstain\" (the abstention format) raised the mean abstention rate from 20% to 91%. Yet within this format, raising t from 0.5 to 0.9 changed it by only -0.8 percentage points.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"Without the abstention-format line, raising the threshold even lowered abstention, and up to 40% of a checkpoint’s items received decisions incompatible with a compliant policy using threshold-independent beliefs.","excerpt":"Without the line, the higher threshold even lowered abstention, and up to 40% of a checkpoint's items received decisions that no compliant policy with threshold-independent beliefs could make.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"The checkpoints barely responded to the threshold in a forced-choice probe.","excerpt":"Given the probability that their answer was correct, the checkpoints barely responded to the threshold in a forced-choice probe.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"Results for an external threshold on recalibrated P(True) varied by calibration and evaluation setting.","excerpt":"Calibrated on the evaluation distribution (post hoc), it beat the abstention format at t = 0.5; calibrated on TriviaQA and applied to PopQA, however, it fell below that format at 0.75.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"},{"text":"Moon recommends reporting abstention rates, responsiveness to the stated threshold, and which items models answer.","excerpt":"evaluations using stated scoring rules should report how often models abstain, whether decisions respond to the stated threshold, and which items they answer.","url":"https://arxiv.org/abs/2611.14661","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14661","checkedAt":"2026-11-13T11:48:07.139Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How often do the tested models’ decisions respond to the stated threshold, and which items do they answer?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","summary":"Dongsun Moon argues that a high abstention rate does not show that an open language model follows a stated scoring rule. Under that rule, a model should answer only when its probability of being correct exceeds a threshold, t. Tests covered 12 checkpoints from three post-training stages of four fully open 3–8B recipes. At t = 0.75, an abstention format raised mean abstention from 20% to 91%. But within that format, changing t from 0.5 to 0.9 changed abstention by only -0.8 percentage points. Without the format, the higher threshold sometimes lowered abstention; some decisions were inconsistent with any compliant policy using threshold-independent beliefs. In a forced-choice probe, the checkpoints barely responded to t. Moon reports that comparisons with self-assessment signals such as P(True) depended on the grader and answer drift between prompts. An external threshold on recalibrated P(True) made coverage non-increasing by construction. Its results differed across evaluation settings: it beat the abstention format when calibrated post hoc on the evaluation distribution, yet fell below it when calibrated on TriviaQA and applied to PopQA at 0.75. Moon concludes that evaluations should report abstention rates, responsiveness to the stated threshold, and which items models answer.","question":"How often do the tested models’ decisions respond to the stated threshold, and which items do they answer?","claims":[{"text":"Moon argues that a high abstention rate alone does not establish compliance with a stated scoring rule.","excerpt":"A high abstention rate is not evidence of compliance","support":"source-stated"},{"text":"At t = 0.75, the abstention format raised mean abstention from 20% to 91%, but changing t from 0.5 to 0.9 within that format changed it by only -0.8 percentage points.","excerpt":"At t = 0.75, a line limiting the reply to an answer or \"I abstain\" (the abstention format) raised the mean abstention rate from 20% to 91%. Yet within this format, raising t from 0.5 to 0.9 changed it by only -0.8 percentage points.","support":"source-stated"},{"text":"Without the abstention-format line, raising the threshold even lowered abstention, and up to 40% of a checkpoint’s items received decisions incompatible with a compliant policy using threshold-independent beliefs.","excerpt":"Without the line, the higher threshold even lowered abstention, and up to 40% of a checkpoint's items received decisions that no compliant policy with threshold-independent beliefs could make.","support":"source-stated"},{"text":"The checkpoints barely responded to the threshold in a forced-choice probe.","excerpt":"Given the probability that their answer was correct, the checkpoints barely responded to the threshold in a forced-choice probe.","support":"source-stated"},{"text":"Results for an external threshold on recalibrated P(True) varied by calibration and evaluation setting.","excerpt":"Calibrated on the evaluation distribution (post hoc), it beat the abstention format at t = 0.5; calibrated on TriviaQA and applied to PopQA, however, it fell below that format at 0.75.","support":"source-stated"},{"text":"Moon recommends reporting abstention rates, responsiveness to the stated threshold, and which items models answer.","excerpt":"evaluations using stated scoring rules should report how often models abstain, whether decisions respond to the stated threshold, and which items they answer.","support":"source-stated"}],"limitations":["The supplied evidence is the paper’s abstract, not its full methods or results; it does not provide details needed to assess the evaluation procedures.","The reported tests cover the specified recipes, checkpoints, and probes; the abstract does not establish how the findings extend beyond them."],"mediaId":"381c12c20a76be4a3b0844df847e93e3609c813ccf37975cdcbb630e52b57007","correctionReason":null,"id":"72c7fe47ecbbff7625e259e939fede5fc6cf4a1cf84b23ce41c47af33a13c836","publicationId":"560a4d1a-cc97-4219-8792-77d67ca73a43","version":1,"sourceUrl":"https://arxiv.org/abs/2611.14661","sourceTitle":"Abstention Is Not Threshold Compliance: Stated Scoring Rules in Open Language Models","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-10","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-13T07:43:10.226Z","createdAt":"2026-11-13T11:17:39.177Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"381c12c20a76be4a3b0844df847e93e3609c813ccf37975cdcbb630e52b57007","kind":"article","url":"https://arxiv.org/abs/2611.14661","sourceUrl":"https://arxiv.org/abs/2611.14661","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-13T11:16:48.955Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.14661","checkedAt":"2026-11-13T11:48:07.139Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v7","mediaId":"381c12c20a76be4a3b0844df847e93e3609c813ccf37975cdcbb630e52b57007","confidence":0.6,"provider":"typesafe","model":"jev-1.13.0","inputHash":"a009658c13f8d8ba8a2ac79eda856e39f77d8233b1848726bb23acab6a70680a","decidedAt":"2026-11-13T11:16:49.377Z"},"provenance":{"textHash":"16d991426c50f9cf38cd6944b74f711656729a00ee6c4aa880ff53e0c22d3b59","observationId":"abbe6aee1ddd3541118ca2ebc7599a0036545c22d3473c1cd2ad368422ad7582","retrievedAt":"2026-11-13T11:16:48.906Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"d9a03e73-16ee-44cf-8c86-69d356581123"},"reviewer":{"model":"gpt-6-luna","runId":"bae02d66-40e3-4b57-81cd-ecdc124f621b"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-13T11:17:39.177Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Moon, Dongsun","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Moon, Dongsun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Moon, Dongsun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.14661","locator":"head/meta[name=author|citation_author]"},"assertionId":"d0acc9a68de62e04cf7aaebe3669786a603149715f887a26c76a76f37be55511:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T11:48:07.139Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"d0acc9a68de62e04cf7aaebe3669786a603149715f887a26c76a76f37be55511","sourceUrl":"https://arxiv.org/abs/2611.14661","contentSha256":"5187e1390ee93c5fa5a7bfae6e7ba4619bb1329e62fe5720fc1dd7a590ac72a9","textHash":"16d991426c50f9cf38cd6944b74f711656729a00ee6c4aa880ff53e0c22d3b59","observedAt":"2026-11-13T11:48:07.139Z"}}],"participationCoverage":{"total":1,"returned":1,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.14661","snapshotId":"d0acc9a68de62e04cf7aaebe3669786a603149715f887a26c76a76f37be55511","contentSha256":"5187e1390ee93c5fa5a7bfae6e7ba4619bb1329e62fe5720fc1dd7a590ac72a9","textHash":"16d991426c50f9cf38cd6944b74f711656729a00ee6c4aa880ff53e0c22d3b59","observedAt":"2026-11-13T11:48:07.139Z","basis":"publisher-page-metadata","sourceLinks":40,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/72c7fe47ecbbff7625e259e939fede5fc6cf4a1cf84b23ce41c47af33a13c836/source"}}}],"audits":[{"publicationId":"560a4d1a-cc97-4219-8792-77d67ca73a43","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft correctly attributes the argument and findings to Dongsun Moon, the paper's sole listed author, on arXiv. One small repair would improve clarity: the scoring-rule proposal comes from Kalai et al., while Moon tests it. The draft says only 'Under that rule' and could add 'proposed by Kalai et al.' This omission does not misattribute any claim.","excerpt":"Kalai et al. propose stating the scoring rule in the prompt with an explicit threshold t"},{"dimension":"summary","status":"uncertain","explanation":"The numbers are faithful, but the reading is hard to follow without specialist background. Central terms are left unexplained: checkpoints, post-training stages, recipes, P(True), coverage, forced-choice probe, TriviaQA/PopQA and post hoc. The draft never says why answer-or-abstain rules matter, such as reducing wrong answers. 'Non-increasing by construction' drops 'in t'. 'Sometimes lowered' softens the source's 'even lowered', and 'some decisions' drops 'up to 40%'. The OLMo 3 7B training results are omitted. The question restates the paper's recommendation, which the abstract already partly answers ('barely responded'). The title only prefixes the publisher.","excerpt":"Without the line, the higher threshold even lowered abstention, and up to 40% of a checkpoint's items received decisions that no compliant policy with threshold-independent beliefs could make."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-10 matches the arXiv v1 submission date. The later feed date of Nov 13 is correctly not used.","excerpt":"[Submitted on 10 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim and excerpt matches the abstract verbatim and stays attributed as Moon's findings. The draft adds no unsupported benefits, costs or forecasts. The limitations correctly note that only the abstract is available and that scope is limited to the tested recipes. The wording 'sometimes lowered' is a slight softening but is not a contradiction.","excerpt":"A high abstention rate is not evidence of compliance: evaluations using stated scoring rules should report how often models abstain, whether decisions respond to the stated threshold, and which items they answer."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified arXiv abstract page, which is the direct original source and the most relevant link available.","excerpt":null}],"createdAt":"2026-11-13T11:17:39.177Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}