# arXiv: Beyond Selected Breakthroughs in Mathematical Conjectures [Canonical reading](https://escapemeridian.com/readings/arxiv-beyond-selected-breakthroughs-in-mathematical-conjectures-f4332b65) Authors Helen Jenne, Davis Brown, Jesse He, and Henry Kvinge argue that studying celebrated AI-assisted mathematics breakthroughs alone gives an incomplete picture of language model capabilities and the computational resources behind reported results. They introduce OpenConjecture, a dataset of conjectures from mathematics papers published on arXiv since February 2026, and test frontier GPT models on a random sample of 200 conjectures over three passes, without problem-specific mathematical guidance. The models produced candidate solutions for roughly 60% of the conjectures; counterexamples were about 1.3 times more common than proofs. The authors did not attempt expert verification, so these results describe candidate solutions, not verified mathematical discoveries. They report that some proposed resolutions fall outside the authors’ intended meaning, some proofs closely follow existing results, and some resolutions show at least moderate novelty. Their argument is that evaluation must look beyond correctness to examine fidelity to the intended statement, attribution, and the nature of the mathematical contribution. How many of the candidate solutions would withstand expert verification? Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv math.HO · Article Original source date: Oct 29, 2026 (2026-10-29). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-11T04:55:37.634Z. First observed: 2026-10-30T06:39:55.414Z. Retrieved: 2026-10-31T04:01:10.919Z. - [Beyond Selected Breakthroughs: Evaluating Language Models on Randomly Sampled Mathematical Conjectures](https://arxiv.org/abs/2610.40382) ## Limits The authors explicitly did not attempt expert verification, so the evidence does not establish which candidate solutions are correct. The reported sample covers 200 conjectures drawn from mathematics papers published on arXiv since February 2026; the abstract does not establish how well these results generalize beyond that sample. ## Open question How many of the candidate solutions would withstand expert verification? ## Source-stated The authors argue that studying only major AI-assisted mathematics breakthroughs gives an incomplete picture of language model capabilities and the computational resources behind reported results. > studying only the current set of major breakthroughs provides an incomplete picture of LLM capabilities and the computational resources needed to achieve reported results. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The paper introduces OpenConjecture, a dataset of conjectures from mathematics papers published on arXiv since February 2026. > we introduce a new dataset, OpenConjecture, which collects conjectures from mathematics papers published on the arXiv since February 2026. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors tested frontier GPT models on a random sample of 200 conjectures over three passes, using a reproducible protocol without problem-specific mathematical guidance. > apply frontier GPT models to a random sample of 200 of these conjectures over three passes using a reproducible protocol without problem-specific mathematical guidance. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The models produced candidate solutions for roughly 60% of the conjectures, and counterexamples were about 1.3 times more common than proofs. > The models we test produce candidate solutions for roughly 60% of the conjectures, with counterexamples about 1.3 times more common than proofs. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors did not attempt expert verification of the candidate solutions, so the reported outcomes do not establish how many are valid solutions. > Rather than attempt expert verification of these candidate solutions, we analyze the types of problems for which LLMs produce solutions and the characteristics of these solutions. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors report examples of resolutions outside the conjectures’ intended meaning, proofs closely following existing results, and resolutions with at least moderate novelty. > models sometimes propose resolutions that are outside the authors' intent (e.g., proposing the trivial group as a counterexample) and that a number of proofs follow closely from existing results, but that some resolutions exhibit at least moderate novelty. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors call for evaluations that assess fidelity to the intended statement, attribution, and the nature of the mathematical contribution, in addition to correctness. > need for evaluations that go beyond correctness to assess fidelity to the intended statement, attribution, and the nature of the mathematical contribution. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source and contributor context Jenne, Helen, Brown, Davis, He, Jesse, Kvinge, Henry · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Nature Nature publishes research, news, and commentary across the sciences. - [Explore this context](/entities/nature) - [Reference](https://www.nature.com/) - [Reference](https://www.nature.com/nature.rss) ## Editorial provenance Version 1. Publication ID: 9d79b3bd-8a96-4df0-8bbc-639862c963eb. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T04:55:37.634Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 145331772ae17d4b8d89f78fbe75a0113ebeaac535824069f06418e3b43d5b83. Source text hash: 05953b479c38e5be9d04c8762376a51d6b1c7e5dde3b58048def723dcccfdd7e. ## Revision and source-check history ### Version 1 — 2026-11-11T04:55:37.634Z Authors Helen Jenne, Davis Brown, Jesse He, and Henry Kvinge argue that studying celebrated AI-assisted mathematics breakthroughs alone gives an incomplete picture of language model capabilities and the computational resources behind reported results. They introduce OpenConjecture, a dataset of conjectures from mathematics papers published on arXiv since February 2026, and test frontier GPT models on a random sample of 200 conjectures over three passes, without problem-specific mathematical guidance. The models produced candidate solutions for roughly 60% of the conjectures; counterexamples were about 1.3 times more common than proofs. The authors did not attempt expert verification, so these results describe candidate solutions, not verified mathematical discoveries. They report that some proposed resolutions fall outside the authors’ intended meaning, some proofs closely follow existing results, and some resolutions show at least moderate novelty. Their argument is that evaluation must look beyond correctness to examine fidelity to the intended statement, attribution, and the nature of the mathematical contribution. How many of the candidate solutions would withstand expert verification? ## Original source arXiv math.HO · Article Original source date: Oct 29, 2026 (2026-10-29). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-11T04:55:37.634Z. First observed: 2026-10-30T06:39:55.414Z. Retrieved: 2026-10-31T04:01:10.919Z. - [Beyond Selected Breakthroughs: Evaluating Language Models on Randomly Sampled Mathematical Conjectures](https://arxiv.org/abs/2610.40382) ## Limits The authors explicitly did not attempt expert verification, so the evidence does not establish which candidate solutions are correct. The reported sample covers 200 conjectures drawn from mathematics papers published on arXiv since February 2026; the abstract does not establish how well these results generalize beyond that sample. ## Open question How many of the candidate solutions would withstand expert verification? ## Source-stated The authors argue that studying only major AI-assisted mathematics breakthroughs gives an incomplete picture of language model capabilities and the computational resources behind reported results. > studying only the current set of major breakthroughs provides an incomplete picture of LLM capabilities and the computational resources needed to achieve reported results. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The paper introduces OpenConjecture, a dataset of conjectures from mathematics papers published on arXiv since February 2026. > we introduce a new dataset, OpenConjecture, which collects conjectures from mathematics papers published on the arXiv since February 2026. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors tested frontier GPT models on a random sample of 200 conjectures over three passes, using a reproducible protocol without problem-specific mathematical guidance. > apply frontier GPT models to a random sample of 200 of these conjectures over three passes using a reproducible protocol without problem-specific mathematical guidance. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The models produced candidate solutions for roughly 60% of the conjectures, and counterexamples were about 1.3 times more common than proofs. > The models we test produce candidate solutions for roughly 60% of the conjectures, with counterexamples about 1.3 times more common than proofs. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors did not attempt expert verification of the candidate solutions, so the reported outcomes do not establish how many are valid solutions. > Rather than attempt expert verification of these candidate solutions, we analyze the types of problems for which LLMs produce solutions and the characteristics of these solutions. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors report examples of resolutions outside the conjectures’ intended meaning, proofs closely following existing results, and resolutions with at least moderate novelty. > models sometimes propose resolutions that are outside the authors' intent (e.g., proposing the trivial group as a counterexample) and that a number of proofs follow closely from existing results, but that some resolutions exhibit at least moderate novelty. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source-stated The authors call for evaluations that assess fidelity to the intended statement, attribution, and the nature of the mathematical contribution, in addition to correctness. > need for evaluations that go beyond correctness to assess fidelity to the intended statement, attribution, and the nature of the mathematical contribution. - [Supporting source](https://arxiv.org/abs/2610.40382) ## Source and contributor context Jenne, Helen, Brown, Davis, He, Jesse, Kvinge, Henry · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Nature Nature publishes research, news, and commentary across the sciences. - [Explore this context](/entities/nature) - [Reference](https://www.nature.com/) - [Reference](https://www.nature.com/nature.rss) ## Editorial provenance Version 1. Publication ID: 9d79b3bd-8a96-4df0-8bbc-639862c963eb. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-11T04:55:37.634Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 145331772ae17d4b8d89f78fbe75a0113ebeaac535824069f06418e3b43d5b83. Source text hash: 05953b479c38e5be9d04c8762376a51d6b1c7e5dde3b58048def723dcccfdd7e. ### uncertain — 2026-11-11T04:55:37.634Z — anthropic/claude-opus-5.5 Publication: 9d79b3bd-8a96-4df0-8bbc-639862c963eb attribution — pass: The reading correctly credits Helen Jenne, Davis Brown, Jesse He and Henry Kvinge. It presents the dataset, experiment, findings and call for broader evaluation as the authors' work and argument. Results are framed as reported candidate solutions, not established facts. > Authors:Helen Jenne, Davis Brown, Jesse He, Henry Kvinge summary — uncertain: The reading is largely faithful and accessible, but one phrase is ambiguous. The summary says some resolutions 'fall outside the authors' intended meaning,' and 'the authors' elsewhere refers to Jenne et al. The source means the authors of the original conjectures. Smallest repair: 'outside the conjecture authors' intended meaning.' Restoring the source's trivial-group example would also help a lay reader. Separately, the title 'Beyond Selected Breakthroughs in Mathematical Conjectures' is not the paper's title and says little about what happened. The question follows naturally from the lack of verification. > models sometimes propose resolutions that are outside the authors' intent (e.g., proposing the trivial group as a counterexample) date — pass: The originalPublished value of 2026-10-29 matches the arXiv v1 submission date. The later feed date was correctly not used. > \[Submitted on 29 Oct 2026\] support — pass: Every claim is grounded in the abstract. The roughly 60% rate, the 1.3x ratio of counterexamples to proofs, the 200-conjecture sample, the three passes and the absence of problem-specific guidance are all stated. The reading correctly declines to treat candidate solutions as verified. Its limitations about the missing verification and the sample's scope are fair, conditional inferences rather than added claims. > Rather than attempt expert verification of these candidate solutions, we analyze the types of problems for which LLMs produce solutions and the characteristics of these solutions. media — pass: The selected media is the verified direct link to the arXiv abstract page of this paper. That page is the actual source and the most direct access point. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/f4332b655105776aa2f044b9c364f819a1b386b9bad82f668be7951c3e7ac7b6/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-beyond-selected-breakthroughs-in-mathematical-conjectures-f4332b65.json)