{"id":"38ca62e34d6be8c483070150883ee83b2e13e9cfe411a7c5599d5927dec2ba74","title":"arXiv cs.AI: Do Language Models Know What Compelling Scientific Evidence Looks Like?","argument":"The authors introduce EDB-Bio, a benchmark built from data in three biological domains, to test how language models assess evidence from multiple studies. In each of its 96k episodes, a model sees a sequence of experimental results and predicts the probability that a target hypothesis is true.\n\nThey report that open-weight and frontier models, without additional training, show systematic underconfidence and achieve Brier scores comparable to a trivial base-rate forecaster. Post-training against strictly proper scoring rules improves calibration and discrimination. In a sequential experiment selection task, their Ibis0 model is reported to be 2.5 times more efficient than the best frontier model.\n\nThe authors also use Bayes factors to examine how models update beliefs, whether their judgments about evidence relevance agree with expert biologists, and where inconsistencies and biases appear. Their results concern the benchmark and experiment selection task described in the abstract; the abstract does not establish how the findings transfer to other scientific domains or real-world discovery. The authors say they release the code, data, and trained models.\n\nWhat conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The available evidence is the paper's abstract; it does not give the methods or conditions needed to assess the reported 2.5-times efficiency comparison. The abstract describes three biological domains and a sequential experiment selection task, but does not establish how the reported results transfer to other scientific domains or real-world discovery.","speaker":{"name":"Igoe, Conor, Grand, Gabriel, Koch, Zane, Bou, Albert, Moss, Eli, Hinks, Michaela M., Rodriques, Samuel G., White, Andrew, Narayanan, Siddharth","roleAtSource":"Article author"},"source":{"title":"Do Language Models Know What Compelling Scientific Evidence Looks Like?","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.41015","publishedAt":"2026-10-29","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce EDB-Bio, a benchmark based on real-world data from three biological domains, with episodes asking models to estimate the probability of a target hypothesis after seeing experimental results.","excerpt":"Each of EDB-Bio's 96k episodes presents a sequence of experimental results from published studies and asks the model to predict the probability that a target hypothesis is true.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"},{"text":"The authors report systematic underconfidence and Brier scores comparable to a trivial base-rate forecaster for open-weight and frontier models without additional training.","excerpt":"Out-of-the-box, open-weight and frontier models alike demonstrate systematic underconfidence and attain Brier scores comparable to a trivial base-rate forecaster.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"},{"text":"The authors report that post-training against strictly proper scoring rules improves calibration and discrimination, and that Ibis0 is more efficient in their sequential experiment selection task.","excerpt":"Post-training against strictly proper scoring rules significantly improves both calibration and discrimination. This calibration pays off downstream: in a sequential experiment selection task, our Ibis0 model is 2.5 times more efficient than the best frontier model.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"},{"text":"The authors use Bayes factors to analyze model belief updates, agreement with expert biologists about evidence relevance, and inconsistencies and biases.","excerpt":"quantifying evidence weights as Bayes factors reveals how models update their beliefs over time, whether they agree with expert biologists on relevance, and where they exhibit inconsistencies and biases.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41015","checkedAt":"2026-11-10T11:50:07.210Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Do Language Models Know What Compelling Scientific Evidence Looks Like?","summary":"The authors introduce EDB-Bio, a benchmark built from data in three biological domains, to test how language models assess evidence from multiple studies. In each of its 96k episodes, a model sees a sequence of experimental results and predicts the probability that a target hypothesis is true.\n\nThey report that open-weight and frontier models, without additional training, show systematic underconfidence and achieve Brier scores comparable to a trivial base-rate forecaster. Post-training against strictly proper scoring rules improves calibration and discrimination. In a sequential experiment selection task, their Ibis0 model is reported to be 2.5 times more efficient than the best frontier model.\n\nThe authors also use Bayes factors to examine how models update beliefs, whether their judgments about evidence relevance agree with expert biologists, and where inconsistencies and biases appear. Their results concern the benchmark and experiment selection task described in the abstract; the abstract does not establish how the findings transfer to other scientific domains or real-world discovery. The authors say they release the code, data, and trained models.\n\nWhat conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?","question":"What conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?","claims":[{"text":"The authors introduce EDB-Bio, a benchmark based on real-world data from three biological domains, with episodes asking models to estimate the probability of a target hypothesis after seeing experimental results.","excerpt":"Each of EDB-Bio's 96k episodes presents a sequence of experimental results from published studies and asks the model to predict the probability that a target hypothesis is true.","support":"source-stated"},{"text":"The authors report systematic underconfidence and Brier scores comparable to a trivial base-rate forecaster for open-weight and frontier models without additional training.","excerpt":"Out-of-the-box, open-weight and frontier models alike demonstrate systematic underconfidence and attain Brier scores comparable to a trivial base-rate forecaster.","support":"source-stated"},{"text":"The authors report that post-training against strictly proper scoring rules improves calibration and discrimination, and that Ibis0 is more efficient in their sequential experiment selection task.","excerpt":"Post-training against strictly proper scoring rules significantly improves both calibration and discrimination. This calibration pays off downstream: in a sequential experiment selection task, our Ibis0 model is 2.5 times more efficient than the best frontier model.","support":"source-stated"},{"text":"The authors use Bayes factors to analyze model belief updates, agreement with expert biologists about evidence relevance, and inconsistencies and biases.","excerpt":"quantifying evidence weights as Bayes factors reveals how models update their beliefs over time, whether they agree with expert biologists on relevance, and where they exhibit inconsistencies and biases.","support":"source-stated"}],"limitations":["The available evidence is the paper's abstract; it does not give the methods or conditions needed to assess the reported 2.5-times efficiency comparison.","The abstract describes three biological domains and a sequential experiment selection task, but does not establish how the reported results transfer to other scientific domains or real-world discovery."],"mediaId":"d2bcc4ef032d07f91ca18ba19bb1cd0b0e4ef147c4f41392410e919f3c1a2091","correctionReason":null,"id":"38ca62e34d6be8c483070150883ee83b2e13e9cfe411a7c5599d5927dec2ba74","publicationId":"9d580cff-87c7-432e-835a-039d2a0b7f84","version":1,"sourceUrl":"https://arxiv.org/abs/2610.41015","sourceTitle":"Do Language Models Know What Compelling Scientific Evidence Looks Like?","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-29","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-02T05:01:23.175Z","createdAt":"2026-11-10T11:34:16.028Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"d2bcc4ef032d07f91ca18ba19bb1cd0b0e4ef147c4f41392410e919f3c1a2091","kind":"article","url":"https://arxiv.org/abs/2610.41015","sourceUrl":"https://arxiv.org/abs/2610.41015","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-02T10:46:17.827Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41015","checkedAt":"2026-11-10T11:50:07.210Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"d2bcc4ef032d07f91ca18ba19bb1cd0b0e4ef147c4f41392410e919f3c1a2091","confidence":0.9,"provider":"typesafe","model":"jev-1.13.0","inputHash":"00702ca6471950d18598d95c430ea289cd20f1d7b143ac32f1185fa05e3ddf44","decidedAt":"2026-11-02T10:46:20.200Z"},"provenance":{"textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observationId":"202bf11c950fa49e01879d52c239c78b93c5ec95c07ddbf87d7ff4d848e14014","retrievedAt":"2026-11-02T10:46:17.526Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"fa88d902-ccf0-40a4-865f-260256b7a634"},"reviewer":{"model":"gpt-6-luna","runId":"ae23d7f7-f481-479a-8f22-257053773f9d"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T11:34:16.028Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Igoe, Conor","url":null,"kind":"unknown"},{"name":"Grand, Gabriel","url":null,"kind":"unknown"},{"name":"Koch, Zane","url":null,"kind":"unknown"},{"name":"Bou, Albert","url":null,"kind":"unknown"},{"name":"Moss, Eli","url":null,"kind":"unknown"},{"name":"Hinks, Michaela M.","url":null,"kind":"unknown"},{"name":"Rodriques, Samuel G.","url":null,"kind":"unknown"},{"name":"White, Andrew","url":null,"kind":"unknown"},{"name":"Narayanan, Siddharth","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Igoe, Conor","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Igoe, Conor","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Grand, Gabriel","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Grand, Gabriel","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Koch, Zane","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Koch, Zane","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Bou, Albert","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bou, Albert","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Moss, Eli","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Moss, Eli","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Hinks, Michaela M.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hinks, Michaela M.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Rodriques, Samuel G.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rodriques, Samuel G.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"White, Andrew","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"White, Andrew","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Narayanan, Siddharth","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Narayanan, Siddharth","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:8","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}}],"participationCoverage":{"total":9,"returned":9,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.41015","snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z","basis":"publisher-page-metadata","sourceLinks":50,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/38ca62e34d6be8c483070150883ee83b2e13e9cfe411a7c5599d5927dec2ba74/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-do-language-models-know-what-compelling-scientific-38ca62e3","formats":{"html":"/readings/arxiv-cs-ai-do-language-models-know-what-compelling-scientific-38ca62e3","markdown":"/readings/arxiv-cs-ai-do-language-models-know-what-compelling-scientific-38ca62e3.md","json":"/readings/arxiv-cs-ai-do-language-models-know-what-compelling-scientific-38ca62e3.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-open-weights","name":"Open weights","explanation":"A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.","referenceUrl":"https://opensource.org/ai/open-weights"},{"id":"term-frontier-model","name":"Frontier model","explanation":"One of the most capable AI models available at a given time; the set shifts as models improve.","referenceUrl":"https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"38ca62e34d6be8c483070150883ee83b2e13e9cfe411a7c5599d5927dec2ba74","revision":"9d580cff-87c7-432e-835a-039d2a0b7f84","revisionKind":"publication-id","title":"arXiv cs.AI: Do Language Models Know What Compelling Scientific Evidence Looks Like?","url":"https://escapemeridian.com/readings/arxiv-cs-ai-do-language-models-know-what-compelling-scientific-38ca62e3"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.41015","revision":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","revisionKind":"sha256-source-text","title":"Do Language Models Know What Compelling Scientific Evidence Looks Like?","url":"https://arxiv.org/abs/2610.41015"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"38ca62e34d6be8c483070150883ee83b2e13e9cfe411a7c5599d5927dec2ba74","title":"arXiv cs.AI: Do Language Models Know What Compelling Scientific Evidence Looks Like?","argument":"The authors introduce EDB-Bio, a benchmark built from data in three biological domains, to test how language models assess evidence from multiple studies. In each of its 96k episodes, a model sees a sequence of experimental results and predicts the probability that a target hypothesis is true.\n\nThey report that open-weight and frontier models, without additional training, show systematic underconfidence and achieve Brier scores comparable to a trivial base-rate forecaster. Post-training against strictly proper scoring rules improves calibration and discrimination. In a sequential experiment selection task, their Ibis0 model is reported to be 2.5 times more efficient than the best frontier model.\n\nThe authors also use Bayes factors to examine how models update beliefs, whether their judgments about evidence relevance agree with expert biologists, and where inconsistencies and biases appear. Their results concern the benchmark and experiment selection task described in the abstract; the abstract does not establish how the findings transfer to other scientific domains or real-world discovery. The authors say they release the code, data, and trained models.\n\nWhat conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The available evidence is the paper's abstract; it does not give the methods or conditions needed to assess the reported 2.5-times efficiency comparison. The abstract describes three biological domains and a sequential experiment selection task, but does not establish how the reported results transfer to other scientific domains or real-world discovery.","speaker":{"name":"Igoe, Conor, Grand, Gabriel, Koch, Zane, Bou, Albert, Moss, Eli, Hinks, Michaela M., Rodriques, Samuel G., White, Andrew, Narayanan, Siddharth","roleAtSource":"Article author"},"source":{"title":"Do Language Models Know What Compelling Scientific Evidence Looks Like?","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.41015","publishedAt":"2026-10-29","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce EDB-Bio, a benchmark based on real-world data from three biological domains, with episodes asking models to estimate the probability of a target hypothesis after seeing experimental results.","excerpt":"Each of EDB-Bio's 96k episodes presents a sequence of experimental results from published studies and asks the model to predict the probability that a target hypothesis is true.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"},{"text":"The authors report systematic underconfidence and Brier scores comparable to a trivial base-rate forecaster for open-weight and frontier models without additional training.","excerpt":"Out-of-the-box, open-weight and frontier models alike demonstrate systematic underconfidence and attain Brier scores comparable to a trivial base-rate forecaster.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"},{"text":"The authors report that post-training against strictly proper scoring rules improves calibration and discrimination, and that Ibis0 is more efficient in their sequential experiment selection task.","excerpt":"Post-training against strictly proper scoring rules significantly improves both calibration and discrimination. This calibration pays off downstream: in a sequential experiment selection task, our Ibis0 model is 2.5 times more efficient than the best frontier model.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"},{"text":"The authors use Bayes factors to analyze model belief updates, agreement with expert biologists about evidence relevance, and inconsistencies and biases.","excerpt":"quantifying evidence weights as Bayes factors reveals how models update their beliefs over time, whether they agree with expert biologists on relevance, and where they exhibit inconsistencies and biases.","url":"https://arxiv.org/abs/2610.41015","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41015","checkedAt":"2026-11-10T11:50:07.210Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Do Language Models Know What Compelling Scientific Evidence Looks Like?","summary":"The authors introduce EDB-Bio, a benchmark built from data in three biological domains, to test how language models assess evidence from multiple studies. In each of its 96k episodes, a model sees a sequence of experimental results and predicts the probability that a target hypothesis is true.\n\nThey report that open-weight and frontier models, without additional training, show systematic underconfidence and achieve Brier scores comparable to a trivial base-rate forecaster. Post-training against strictly proper scoring rules improves calibration and discrimination. In a sequential experiment selection task, their Ibis0 model is reported to be 2.5 times more efficient than the best frontier model.\n\nThe authors also use Bayes factors to examine how models update beliefs, whether their judgments about evidence relevance agree with expert biologists, and where inconsistencies and biases appear. Their results concern the benchmark and experiment selection task described in the abstract; the abstract does not establish how the findings transfer to other scientific domains or real-world discovery. The authors say they release the code, data, and trained models.\n\nWhat conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?","question":"What conditions and efficiency measure underlie the reported comparison between Ibis0 and the best frontier model in the sequential experiment selection task?","claims":[{"text":"The authors introduce EDB-Bio, a benchmark based on real-world data from three biological domains, with episodes asking models to estimate the probability of a target hypothesis after seeing experimental results.","excerpt":"Each of EDB-Bio's 96k episodes presents a sequence of experimental results from published studies and asks the model to predict the probability that a target hypothesis is true.","support":"source-stated"},{"text":"The authors report systematic underconfidence and Brier scores comparable to a trivial base-rate forecaster for open-weight and frontier models without additional training.","excerpt":"Out-of-the-box, open-weight and frontier models alike demonstrate systematic underconfidence and attain Brier scores comparable to a trivial base-rate forecaster.","support":"source-stated"},{"text":"The authors report that post-training against strictly proper scoring rules improves calibration and discrimination, and that Ibis0 is more efficient in their sequential experiment selection task.","excerpt":"Post-training against strictly proper scoring rules significantly improves both calibration and discrimination. This calibration pays off downstream: in a sequential experiment selection task, our Ibis0 model is 2.5 times more efficient than the best frontier model.","support":"source-stated"},{"text":"The authors use Bayes factors to analyze model belief updates, agreement with expert biologists about evidence relevance, and inconsistencies and biases.","excerpt":"quantifying evidence weights as Bayes factors reveals how models update their beliefs over time, whether they agree with expert biologists on relevance, and where they exhibit inconsistencies and biases.","support":"source-stated"}],"limitations":["The available evidence is the paper's abstract; it does not give the methods or conditions needed to assess the reported 2.5-times efficiency comparison.","The abstract describes three biological domains and a sequential experiment selection task, but does not establish how the reported results transfer to other scientific domains or real-world discovery."],"mediaId":"d2bcc4ef032d07f91ca18ba19bb1cd0b0e4ef147c4f41392410e919f3c1a2091","correctionReason":null,"id":"38ca62e34d6be8c483070150883ee83b2e13e9cfe411a7c5599d5927dec2ba74","publicationId":"9d580cff-87c7-432e-835a-039d2a0b7f84","version":1,"sourceUrl":"https://arxiv.org/abs/2610.41015","sourceTitle":"Do Language Models Know What Compelling Scientific Evidence Looks Like?","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-29","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-02T05:01:23.175Z","createdAt":"2026-11-10T11:34:16.028Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"d2bcc4ef032d07f91ca18ba19bb1cd0b0e4ef147c4f41392410e919f3c1a2091","kind":"article","url":"https://arxiv.org/abs/2610.41015","sourceUrl":"https://arxiv.org/abs/2610.41015","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-02T10:46:17.827Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41015","checkedAt":"2026-11-10T11:50:07.210Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"d2bcc4ef032d07f91ca18ba19bb1cd0b0e4ef147c4f41392410e919f3c1a2091","confidence":0.9,"provider":"typesafe","model":"jev-1.13.0","inputHash":"00702ca6471950d18598d95c430ea289cd20f1d7b143ac32f1185fa05e3ddf44","decidedAt":"2026-11-02T10:46:20.200Z"},"provenance":{"textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observationId":"202bf11c950fa49e01879d52c239c78b93c5ec95c07ddbf87d7ff4d848e14014","retrievedAt":"2026-11-02T10:46:17.526Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"fa88d902-ccf0-40a4-865f-260256b7a634"},"reviewer":{"model":"gpt-6-luna","runId":"ae23d7f7-f481-479a-8f22-257053773f9d"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T11:34:16.028Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Igoe, Conor","url":null,"kind":"unknown"},{"name":"Grand, Gabriel","url":null,"kind":"unknown"},{"name":"Koch, Zane","url":null,"kind":"unknown"},{"name":"Bou, Albert","url":null,"kind":"unknown"},{"name":"Moss, Eli","url":null,"kind":"unknown"},{"name":"Hinks, Michaela M.","url":null,"kind":"unknown"},{"name":"Rodriques, Samuel G.","url":null,"kind":"unknown"},{"name":"White, Andrew","url":null,"kind":"unknown"},{"name":"Narayanan, Siddharth","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Igoe, Conor","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Igoe, Conor","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Grand, Gabriel","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Grand, Gabriel","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Koch, Zane","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Koch, Zane","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Bou, Albert","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Bou, Albert","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Moss, Eli","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Moss, Eli","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Hinks, Michaela M.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hinks, Michaela M.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Rodriques, Samuel G.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Rodriques, Samuel G.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"White, Andrew","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"White, Andrew","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}},{"actor":{"name":"Narayanan, Siddharth","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Narayanan, Siddharth","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41015","locator":"head/meta[name=author|citation_author]"},"assertionId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7:participation:8","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T11:50:07.210Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","sourceUrl":"https://arxiv.org/abs/2610.41015","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z"}}],"participationCoverage":{"total":9,"returned":9,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.41015","snapshotId":"485d2f3c8844a3b02739479d6786b6bc2faa5b5e6a776049a63033b4278d34c7","contentSha256":"21a2fe15c31dc4095b9a05f90db18300321cfeeaba150d6c09285062aa0e6779","textHash":"a6313d2ad3e2e0fedad891eb8c6e067b8282672f47a41d0809918e0234fd08aa","observedAt":"2026-11-10T11:50:07.210Z","basis":"publisher-page-metadata","sourceLinks":50,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/38ca62e34d6be8c483070150883ee83b2e13e9cfe411a7c5599d5927dec2ba74/source"}}}],"audits":[{"publicationId":"9d580cff-87c7-432e-835a-039d2a0b7f84","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes the benchmark, results, the Ibis0 comparison and the release to the paper's authors, using hedges such as 'They report' and 'is reported to be'. The arXiv cs.AI label and paper title match the source.","excerpt":"Authors:Conor Igoe, Gabriel Grand, Zane Koch, Albert Bou, Eli Moss, Michaela M. Hinks, Samuel G. Rodriques, Andrew White, Siddharth Narayanan"},{"dimension":"summary","status":"uncertain","explanation":"The reading is faithful, but its central findings depend on unexplained terms: Brier scores, trivial base-rate forecaster, calibration, discrimination, strictly proper scoring rules, Bayes factors and open-weight/frontier. A general reader cannot tell that the untrained models scored about as well as a baseline that always predicts the base rate. It also omits the stated stakes, that language models increasingly drive autonomous discovery systems, and does not say Ibis0 is the authors' post-trained model. Smallest repair: add that motivation, identify Ibis0, and link the key terms to tooltips or gloss them briefly.","excerpt":"To what extent are language models, which are increasingly driving autonomous discovery systems, capable of such evidence deliberation?"},{"dimension":"date","status":"pass","explanation":"The originalPublished value of 2026-10-29 matches the v1 submission date shown on the arXiv page. The later feed date, 2026-11-02, is correctly not used.","excerpt":"[Submitted on 29 Oct 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim matches the abstract, and the excerpts are verbatim. Hedged wording ('reported', 'without additional training') stays within the source. The limitations and the question correctly note what the abstract leaves unestablished. No benefits or costs are invented.","excerpt":"in a sequential experiment selection task, our Ibis0 model is 2.5 times more efficient than the best frontier model."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified, direct link to the arXiv abstract page for this paper, which is the original source.","excerpt":null}],"createdAt":"2026-11-10T11:34:16.028Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}