Ai, Lun, Wu, Yu-Heng, Schaul, Jonathan, Küchenhoff, Leonie, Turei, Denes, Carli, Francesco, Petroni, Fabio, Dugourd, Aurelien, Saez-Rodriguez, Julio · Oct 29, 2026
Researchers test how agents build evidence to verify scientific consensus claims
Markdown · JSON · Original source
The authors argue that scientific agents can better verify whether research literature supports or refutes a claim by maintaining a structured evidence state outside the model’s context, rather than accumulating evidence only within that context. They note that these agents are usually evaluated end-to-end without ground truth, leaving the effects of individual design choices unknown. They study two choices: what information the state contains and how a judge decides when enough evidence has been gathered. Their ProClaim instrument keeps this state separate so the choices can be examined while holding the other fixed. The authors evaluate the approach on ProClaim-eval, a dataset of 718 claims with expert-curated SUPPORT and REFUTE labels from five databases in molecular biology, nuclear physics and astronomy. They report that the evidence state improves performance over in-context accumulation, and that what is stored matters. A trained judge that reads the state performs better than fixed-round stopping. A separately trained judge achieves the highest agreement with gold labels on three of five subsets, matching the off-the-shelf Sonnet 4.6 judge and outperforming Claude Haiku 4.5. The authors conclude that recursively maintaining a structured evidence state outside the model’s context is effective for this task.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 29, 2026 (2026-10-29).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T02:55:39.483Z. First observed: 2026-11-02T05:01:45.269Z. Retrieved: 2026-11-02T11:48:47.472Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported evaluation uses 718 claims with gold labels from five expert-curated databases in molecular biology, nuclear physics and astronomy; the abstract does not report results beyond this dataset. The abstract reports the separately trained judge’s highest agreement on three of five subsets, not all five.
Open question
How would the reported results change when ProClaim is evaluated on consensus claims beyond the five expert-curated databases?
Source-stated
The paper examines what a scientific agent recursively builds as an evidence state and what controls when it stops gathering evidence.
we examined two design axes: (1) what the recursion builds (the state of evidence) and (2) what controls it (the judge that decides when to stop).
Source-stated
ProClaim-eval contains 718 claims with SUPPORT and REFUTE gold labels from five expert-curated databases across three scientific fields.
including 718 claims with SUPPORT and REFUTE gold labels from five expert-curated databases spanning molecular biology, nuclear physics, and astronomy.
Source-stated
The authors report that maintaining an evidence state improves performance over accumulating evidence in the model’s context, and that the contents of the state matter.
We found that the evidence state improves performance over in-context accumulation, and it also matters what is stored in it.
Source-stated
The authors report that a trained judge reading the evidence state performs better than stopping after a fixed number of rounds.
a trained judge that reads the evidence state performs better than fixed-round stopping.
Source-stated
A separately trained judge achieved the highest agreement with gold labels on three of five subsets, matching the Claude Sonnet 4.6 judge and outperforming Claude Haiku 4.5.
A judge trained separately and never exposed to ProClaim-eval achieves the highest agreement with the gold labels on three of five subsets, where it matches the off-the-shelf Claude Sonnet~4.6 judge and outperforms Claude Haiku~4.5.
Source-stated
The authors conclude that recursively maintaining a structured evidence state outside the model’s context is effective for verifying scientific claims in their task.
Our results support recursively maintaining a structured evidence state outside the model's context as an effective design for verifying claims in scientific agents.
Source and contributor context
Ai, Lun, Wu, Yu-Heng, Schaul, Jonathan, Küchenhoff, Leonie, Turei, Denes, Carli, Francesco, Petroni, Fabio, Dugourd, Aurelien, Saez-Rodriguez, Julio · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: c163a282-824d-42e5-8bd8-8202690d2fba.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-11T02:55:39.483Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 38728a90d992fc6dd6ccc58ba37f679644385ee4dcdff622c2780275e0ba4d34. Source text hash: 31deba25b85a7459be20853d7b1a37ec655d3a7a76730b893c50ac66afe4b365.
Revision and source-check history
Version 1 · 2026-11-11T02:55:39.483Z
Researchers test how agents build evidence to verify scientific consensus claims
The authors argue that scientific agents can better verify whether research literature supports or refutes a claim by maintaining a structured evidence state outside the model’s context, rather than accumulating evidence only within that context. They note that these agents are usually evaluated end-to-end without ground truth, leaving the effects of individual design choices unknown. They study two choices: what information the state contains and how a judge decides when enough evidence has been gathered. Their ProClaim instrument keeps this state separate so the choices can be examined while holding the other fixed. The authors evaluate the approach on ProClaim-eval, a dataset of 718 claims with expert-curated SUPPORT and REFUTE labels from five databases in molecular biology, nuclear physics and astronomy. They report that the evidence state improves performance over in-context accumulation, and that what is stored matters. A trained judge that reads the state performs better than fixed-round stopping. A separately trained judge achieves the highest agreement with gold labels on three of five subsets, matching the off-the-shelf Sonnet 4.6 judge and outperforming Claude Haiku 4.5. The authors conclude that recursively maintaining a structured evidence state outside the model’s context is effective for this task.
Original source
cs.AI · Article
Source details
Original source date: Oct 29, 2026 (2026-10-29).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-11T02:55:39.483Z. First observed: 2026-11-02T05:01:45.269Z. Retrieved: 2026-11-02T11:48:47.472Z.
Limits
The reported evaluation uses 718 claims with gold labels from five expert-curated databases in molecular biology, nuclear physics and astronomy; the abstract does not report results beyond this dataset. The abstract reports the separately trained judge’s highest agreement on three of five subsets, not all five.
Open question
How would the reported results change when ProClaim is evaluated on consensus claims beyond the five expert-curated databases?
Source-stated
The paper examines what a scientific agent recursively builds as an evidence state and what controls when it stops gathering evidence.
we examined two design axes: (1) what the recursion builds (the state of evidence) and (2) what controls it (the judge that decides when to stop).
Source-stated
ProClaim-eval contains 718 claims with SUPPORT and REFUTE gold labels from five expert-curated databases across three scientific fields.
including 718 claims with SUPPORT and REFUTE gold labels from five expert-curated databases spanning molecular biology, nuclear physics, and astronomy.
Source-stated
The authors report that maintaining an evidence state improves performance over accumulating evidence in the model’s context, and that the contents of the state matter.
We found that the evidence state improves performance over in-context accumulation, and it also matters what is stored in it.
Source-stated
The authors report that a trained judge reading the evidence state performs better than stopping after a fixed number of rounds.
a trained judge that reads the evidence state performs better than fixed-round stopping.
Source-stated
A separately trained judge achieved the highest agreement with gold labels on three of five subsets, matching the Claude Sonnet 4.6 judge and outperforming Claude Haiku 4.5.
A judge trained separately and never exposed to ProClaim-eval achieves the highest agreement with the gold labels on three of five subsets, where it matches the off-the-shelf Claude Sonnet~4.6 judge and outperforms Claude Haiku~4.5.
Source-stated
The authors conclude that recursively maintaining a structured evidence state outside the model’s context is effective for verifying scientific claims in their task.
Our results support recursively maintaining a structured evidence state outside the model's context as an effective design for verifying claims in scientific agents.
Source and contributor context
Ai, Lun, Wu, Yu-Heng, Schaul, Jonathan, Küchenhoff, Leonie, Turei, Denes, Carli, Francesco, Petroni, Fabio, Dugourd, Aurelien, Saez-Rodriguez, Julio · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Claude
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Editorial provenance
Version 1. Publication ID: c163a282-824d-42e5-8bd8-8202690d2fba.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-11T02:55:39.483Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 38728a90d992fc6dd6ccc58ba37f679644385ee4dcdff622c2780275e0ba4d34. Source text hash: 31deba25b85a7459be20853d7b1a37ec655d3a7a76730b893c50ac66afe4b365.
passed · 2026-11-11T02:55:39.483Z · anthropic/claude-opus-5.5
Publication: c163a282-824d-42e5-8bd8-8202690d2fba
attribution — pass: The draft consistently attributes the findings and the conclusion to the paper's authors, using wording such as 'The authors argue', 'They report' and 'The authors conclude'. It does not present the results as established fact or as Escape Meridian's view. The paper is correctly identified as the arXiv submission by Lun Ai and colleagues.
Authors:Lun Ai, Yu-Heng Wu, Jonathan Schaul, Leonie Küchenhoff, Denes Turei, Francesco Carli, Fabio Petroni, Aurelien Dugourd, Julio Saez-Rodriguez
summary — pass: The title, summary and question faithfully reflect the abstract. The summary covers the two design axes, the ProClaim instrument, the 718-claim dataset and the main results, and it explains consensus verification in plain terms. Two details are compressed but not distorted. First, the draft omits that ProClaim ranks among the top three against verification systems and agent harnesses. Second, it omits that the separate judge was 'never exposed to ProClaim-eval'. Adding that phrase would better preserve the generalization point.
A judge trained separately and never exposed to ProClaim-eval achieves the highest agreement with the gold labels on three of five subsets
date — pass: The originalPublished value of 2026-10-29 matches the arXiv v1 submission date. It is not the later feed date of 2026-11-02.
[Submitted on 29 Oct 2026]
support — pass: Each claim matches a verbatim excerpt from the abstract. The limitations correctly note the dataset scope and the three-of-five-subsets result. The question about generalizing beyond the five databases follows from that stated scope rather than implying an answer. The draft adds no unsupported benefits, costs or forecasts.
Our results support recursively maintaining a structured evidence state outside the model's context as an effective design for verifying claims in scientific agents.
media — pass: The selected media is the verified direct link to the arXiv abstract page, which is the original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Claude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivClaude is Anthropic’s family of AI models and assistants for tasks such as writing, analysis and coding.
Explore ClaudearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv