Ospanov, Azim, Feng, Zijin, Ding, Ding, Liu, Chengwu, Bai, Haoli, Sun, Jiacheng, Shang, Lifeng, Farnia, Farzan · Oct 27, 2026
Can LLMs Reliably Grade Olympiad Proofs? A Study of Mathematical Verification with LLMs
Markdown · JSON · Original source
The authors of this paper study whether can grade Olympiad-level mathematical proofs, where each claim in a long chain of reasoning needs careful assessment. They introduce a human-curated dataset of USAMO 2025 candidate solutions with expert grading and evaluate existing LLM verifiers across . They also propose TROJAN, a process for generating adversarial proofs, and MABGrader, which treats grading as a choice among possible scores. The authors report that MABGrader outperforms ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%. As an Escape Meridian interpretation, these results suggest a possible way to improve automated grading, but the abstract does not establish how well the approach would work beyond the evaluated setting. How does MABGrader perform on proofs outside the USAMO 2025 dataset?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-05T00:02:33.938Z. First observed: 2026-10-28T05:52:45.748Z. Retrieved: 2026-10-30T03:31:37.835Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence is the paper's abstract and does not provide details of the evaluation results or the dataset beyond its description. The abstract does not report how MABGrader performs on proofs outside the USAMO 2025 dataset.
Open question
How does MABGrader perform on proofs outside the USAMO 2025 dataset?
Source-stated
The authors study LLM verification of Olympiad-level proofs, where long reasoning chains require assessment of each claim.
Olympiad-level proofs, which require meticulous assessment of each claim in a long reasoning chain.
Source-stated
The authors introduce a human-curated dataset of USAMO 2025 candidate solutions with expert grading.
a human-curated dataset of USAMO 2025 candidate solutions with expert grading
Source-stated
The authors propose TROJAN, an iterative self-critique pipeline for generating adversarial proofs at scale.
TROJAN, an iterative self-critique pipeline for generating high-fidelity adversarial proofs at scale.
Source-stated
The authors propose MABGrader, which frames grading as selecting an arm over a discrete score space.
MABGrader, a Multi-Armed Bandit framework that reframes grading as arm selection over the discrete score space.
Source-stated
The authors report that MABGrader outperforms ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%.
MABGrader outperforms the SOTA ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%.
Escape Meridian interpretation
The reported results suggest a possible route to improving automated proof grading, but do not establish performance outside the evaluated setting.
state-of-the-art models still leave considerable room for improvement. To narrow this gap, we propose MABGrader
Source and contributor context
Ospanov, Azim, Feng, Zijin, Ding, Ding, Liu, Chengwu, Bai, Haoli, Sun, Jiacheng, Shang, Lifeng, Farnia, Farzan · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 2d0bffe0-c890-4e05-8375-2fba49fd71b1.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-05T00:02:33.938Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 474735c627c126b075da04458a5ea1250d8b8921382f97c34cc6ef4e22483250. Source text hash: f6194b3b17d23802977b0311a80b92d12819df577fdf6c3b52862d3bd94cf658.
Revision and source-check history
Version 1 · 2026-11-05T00:02:33.938Z
Can LLMs Reliably Grade Olympiad Proofs? A Study of Mathematical Verification with LLMs
The authors of this paper study whether can grade Olympiad-level mathematical proofs, where each claim in a long chain of reasoning needs careful assessment. They introduce a human-curated dataset of USAMO 2025 candidate solutions with expert grading and evaluate existing LLM verifiers across . They also propose TROJAN, a process for generating adversarial proofs, and MABGrader, which treats grading as a choice among possible scores. The authors report that MABGrader outperforms ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%. As an Escape Meridian interpretation, these results suggest a possible way to improve automated grading, but the abstract does not establish how well the approach would work beyond the evaluated setting. How does MABGrader perform on proofs outside the USAMO 2025 dataset?
Original source
cs.AI · Article
Source details
Original source date: Oct 27, 2026 (2026-10-27).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-05T00:02:33.938Z. First observed: 2026-10-28T05:52:45.748Z. Retrieved: 2026-10-30T03:31:37.835Z.
Limits
The supplied evidence is the paper's abstract and does not provide details of the evaluation results or the dataset beyond its description. The abstract does not report how MABGrader performs on proofs outside the USAMO 2025 dataset.
Open question
How does MABGrader perform on proofs outside the USAMO 2025 dataset?
Source-stated
The authors study LLM verification of Olympiad-level proofs, where long reasoning chains require assessment of each claim.
Olympiad-level proofs, which require meticulous assessment of each claim in a long reasoning chain.
Source-stated
The authors introduce a human-curated dataset of USAMO 2025 candidate solutions with expert grading.
a human-curated dataset of USAMO 2025 candidate solutions with expert grading
Source-stated
The authors propose TROJAN, an iterative self-critique pipeline for generating adversarial proofs at scale.
TROJAN, an iterative self-critique pipeline for generating high-fidelity adversarial proofs at scale.
Source-stated
The authors propose MABGrader, which frames grading as selecting an arm over a discrete score space.
MABGrader, a Multi-Armed Bandit framework that reframes grading as arm selection over the discrete score space.
Source-stated
The authors report that MABGrader outperforms ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%.
MABGrader outperforms the SOTA ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%.
Escape Meridian interpretation
The reported results suggest a possible route to improving automated proof grading, but do not establish performance outside the evaluated setting.
state-of-the-art models still leave considerable room for improvement. To narrow this gap, we propose MABGrader
Source and contributor context
Ospanov, Azim, Feng, Zijin, Ding, Ding, Liu, Chengwu, Bai, Haoli, Sun, Jiacheng, Shang, Lifeng, Farnia, Farzan · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 2d0bffe0-c890-4e05-8375-2fba49fd71b1.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-05T00:02:33.938Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 474735c627c126b075da04458a5ea1250d8b8921382f97c34cc6ef4e22483250. Source text hash: f6194b3b17d23802977b0311a80b92d12819df577fdf6c3b52862d3bd94cf658.
uncertain · 2026-11-05T00:02:33.938Z · anthropic/claude-opus-5.5
Publication: 2d0bffe0-c890-4e05-8375-2fba49fd71b1
attribution — pass: The reading attributes the dataset, the evaluation, TROJAN, MABGrader and the reported gains to the paper's authors. It labels the closing inference as Escape Meridian interpretation, consistent with the arXiv abstract.
Authors:Azim Ospanov, Zijin Feng, Ding Ding, Chengwu Liu, Haoli Bai, Jiacheng Sun, Lifeng Shang, Farzan Farnia
summary — uncertain: Several central terms are left unexplained for general readers: USAMO (a US national math olympiad), Quadratic Weighted Kappa (an agreement score), and adversarial proofs. Suggest term-library tooltips for these. The summary also swaps the exact term Multi-Armed Bandit for a loose paraphrase. It drops the SOTA label on ProofGrader. It omits two key points: the finding that current frontier models leave considerable room for improvement, and the stated motivation that expert inspection is costly to scale. Smallest repair: restore these points and the exact term. The question also repeats verbatim at the end of the summary.
Current proof verification thus primarily relies on expert inspection, which is costly to scale.
date — pass: originalPublished is 2026-10-27, which matches the v1 submission date on the arXiv page. The 2026-10-29 feed date is correctly not used.
[Submitted on 27 Oct 2026]
support — uncertain: The source-stated claims match the abstract, and the interpretation is labeled and hedged. However, the question and limitations assume MABGrader's 26.4% and 14% gains were measured on the USAMO 2025 dataset. The abstract does not say which data the MABGrader comparison used. The phrase 'outside the USAMO 2025 dataset' could therefore misframe the evaluated setting. Smallest repair: say 'outside the evaluated setting'.
MABGrader outperforms the SOTA ProofGrader by 26.4% in Quadratic Weighted Kappa and reduces grading error by 14%.
media — pass: The selected media is the verified direct link to the original arXiv abstract page. This is the most direct access to the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A language model with many learned numerical settings, used to process and generate language.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA language model with many learned numerical settings, used to process and generate language.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv