# arXiv cs.AI: FrontierPhysics: Evaluating Agents for End-to-End Physics Research [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-frontierphysics-evaluating-agents-for-end-to-end-5d50891b) The authors introduce FrontierPhysics, a benchmark designed to test AI agents across the stages of physics research rather than on fixed questions or isolated tasks. Its 56 tasks, created by 42 physicists from their own research, span nine domains and require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts. The authors combine deterministic verifiers with expert-authored rubrics to assess correctness and whether results are publishable. Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts. The authors say manual review of agent trajectories points to insufficiently rigorous reasoning and limited expert intuition—not gaps in physics knowledge or execution errors—as the main weaknesses of frontier agents. Escape Meridian's reading is that this distinction matters: completing research steps and knowing physics may not be enough to meet the benchmark’s standard for research quality. The abstract does not establish how performance on these tasks translates to work in physics research beyond the benchmark. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T11:26:06.775Z. First observed: 2026-11-05T05:00:33.388Z. Retrieved: 2026-11-09T11:25:17.265Z. - [FrontierPhysics: Evaluating Agents for End-to-End Physics Research](https://arxiv.org/abs/2611.06910) ## Limits The evidence supplied is the arXiv abstract and page text, not the full paper; it does not provide the rubric details or explain how publishability is judged. The abstract reports benchmark results but does not establish how they translate to physics research beyond the benchmark. ## Open question How do the authors determine, through their expert-authored rubrics, whether an agent’s research is publishable? ## Source-stated FrontierPhysics evaluates agents on 56 research tasks created by 42 physicists, covering nine domains and requiring multiple stages of research. > an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated Tasks require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts. > Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated The benchmark combines deterministic verifiers and expert-authored rubrics to evaluate correctness and publishability. > Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts. > Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated The authors identify insufficiently rigorous reasoning and limited expert intuition as frontier agents’ main weaknesses, rather than physics knowledge gaps or execution errors. > frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Escape Meridian interpretation The benchmark results suggest that completing research steps and having physics knowledge may not suffice to meet this benchmark’s standard for research quality. > existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source and contributor context You, Bingran, Ke, Wenjun, Rosenauer, Ria, Liu, Bo, Chen, Wenbo, Chen, Min, Xu, Shuqi, Ma, Yue, Feng, Jiechao, Chen, Bo, Sun, Xiangkai, Liu, Jiarui, Guo, Jinen, Zhou, Zeyu, Huang, Xu, Li, Zhuohan, Li, Haoyu, Lin, Heguang, Ding, Lijie, Zeng, Yadong, Zhang, Kangqi, Liu, Xinge, Liu, Hanwen, Li, Zhenghong, von Recum, Alexander, Zhu, Yuchen, Schwarzfischer, Felix, Zhao, Yizhou, Wang, Jicheng, Li, Binxu, Yu, Qian, Li, Chan, Dai, Cheng, Wang, Di, Zhang, Haoyu, Sun, Jiankai, Feng, Kaituo, Qin, Mian, Yin, Ruihong, Luo, Yiqun, Wu, Weiwei, Pan, Shihong, Hsieh, Tsung Yeh, Huang, Xiao, Zhang, Yan, Li, Sihan, Ren, Zirui, Tu, Zengji, Choe, Kyoung Whan, Sun, Ke, Galia, Vincenzo, Mishra, Vaisakhi, He, Jiashu, Wang, Hongyang, Yang, Zhiheng, Gaur, Shubham, Gu, Yifan, Wang, Shuyi, Zuo, Zihao, Dillmann, Steven, Sun, Yiyou, Han, Xinyang, Lee, Hanchung, Tao, Molei, Zhao, Yue, Liu, Shilong, Wu, Yuchen, Zhao, Xuandong, Li, Xiangyi, Brand, Florian, Ceder, Gerbrand, Haeffner, Hartmut · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 5d943385-64cc-4b79-844e-a1923038164e. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:26:06.775Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 2972272e69ed0df6350dbfc1e93e0d43368cd4dd18279899684218a411d66829. Source text hash: 4cd1efe5bad459af3ec03bf9b3f487d3ceea3d737fe235fcb6429853930b4291. ## Revision and source-check history ### Version 1 — 2026-11-09T11:26:06.775Z The authors introduce FrontierPhysics, a benchmark designed to test AI agents across the stages of physics research rather than on fixed questions or isolated tasks. Its 56 tasks, created by 42 physicists from their own research, span nine domains and require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts. The authors combine deterministic verifiers with expert-authored rubrics to assess correctness and whether results are publishable. Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts. The authors say manual review of agent trajectories points to insufficiently rigorous reasoning and limited expert intuition—not gaps in physics knowledge or execution errors—as the main weaknesses of frontier agents. Escape Meridian's reading is that this distinction matters: completing research steps and knowing physics may not be enough to meet the benchmark’s standard for research quality. The abstract does not establish how performance on these tasks translates to work in physics research beyond the benchmark. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T11:26:06.775Z. First observed: 2026-11-05T05:00:33.388Z. Retrieved: 2026-11-09T11:25:17.265Z. - [FrontierPhysics: Evaluating Agents for End-to-End Physics Research](https://arxiv.org/abs/2611.06910) ## Limits The evidence supplied is the arXiv abstract and page text, not the full paper; it does not provide the rubric details or explain how publishability is judged. The abstract reports benchmark results but does not establish how they translate to physics research beyond the benchmark. ## Open question How do the authors determine, through their expert-authored rubrics, whether an agent’s research is publishable? ## Source-stated FrontierPhysics evaluates agents on 56 research tasks created by 42 physicists, covering nine domains and requiring multiple stages of research. > an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated Tasks require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts. > Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated The benchmark combines deterministic verifiers and expert-authored rubrics to evaluate correctness and publishability. > Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts. > Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source-stated The authors identify insufficiently rigorous reasoning and limited expert intuition as frontier agents’ main weaknesses, rather than physics knowledge gaps or execution errors. > frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Escape Meridian interpretation The benchmark results suggest that completing research steps and having physics knowledge may not suffice to meet this benchmark’s standard for research quality. > existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness. - [Supporting source](https://arxiv.org/abs/2611.06910) ## Source and contributor context You, Bingran, Ke, Wenjun, Rosenauer, Ria, Liu, Bo, Chen, Wenbo, Chen, Min, Xu, Shuqi, Ma, Yue, Feng, Jiechao, Chen, Bo, Sun, Xiangkai, Liu, Jiarui, Guo, Jinen, Zhou, Zeyu, Huang, Xu, Li, Zhuohan, Li, Haoyu, Lin, Heguang, Ding, Lijie, Zeng, Yadong, Zhang, Kangqi, Liu, Xinge, Liu, Hanwen, Li, Zhenghong, von Recum, Alexander, Zhu, Yuchen, Schwarzfischer, Felix, Zhao, Yizhou, Wang, Jicheng, Li, Binxu, Yu, Qian, Li, Chan, Dai, Cheng, Wang, Di, Zhang, Haoyu, Sun, Jiankai, Feng, Kaituo, Qin, Mian, Yin, Ruihong, Luo, Yiqun, Wu, Weiwei, Pan, Shihong, Hsieh, Tsung Yeh, Huang, Xiao, Zhang, Yan, Li, Sihan, Ren, Zirui, Tu, Zengji, Choe, Kyoung Whan, Sun, Ke, Galia, Vincenzo, Mishra, Vaisakhi, He, Jiashu, Wang, Hongyang, Yang, Zhiheng, Gaur, Shubham, Gu, Yifan, Wang, Shuyi, Zuo, Zihao, Dillmann, Steven, Sun, Yiyou, Han, Xinyang, Lee, Hanchung, Tao, Molei, Zhao, Yue, Liu, Shilong, Wu, Yuchen, Zhao, Xuandong, Li, Xiangyi, Brand, Florian, Ceder, Gerbrand, Haeffner, Hartmut · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## AI agent An AI system that chooses and carries out actions toward a goal, often using software tools. - [Definition reference](https://www.ibm.com/think/topics/ai-agents) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 5d943385-64cc-4b79-844e-a1923038164e. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:26:06.775Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 2972272e69ed0df6350dbfc1e93e0d43368cd4dd18279899684218a411d66829. Source text hash: 4cd1efe5bad459af3ec03bf9b3f487d3ceea3d737fe235fcb6429853930b4291. ### uncertain — 2026-11-09T11:26:06.775Z — anthropic/claude-opus-5.5 Publication: 5d943385-64cc-4b79-844e-a1923038164e attribution — pass: The reading is attributed to the arXiv cs.AI paper and its authors. Benchmark design, results and the diagnosis of agent weaknesses are framed as what the authors introduce or say. The Escape Meridian inference is explicitly labeled as EM's reading. > We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research summary — pass: The summary leads with what was introduced and why: existing benchmarks test fixed questions or isolated sub-tasks. It accurately conveys task scope, evaluation method, headline result and diagnosed weaknesses. The question is a genuine open issue the abstract leaves unresolved. Terms such as rollouts, configurations, deterministic verifiers and agent trajectories are kept exactly as the source uses them. They could use term-library tooltips; for example, rollout means an attempt. The central finding remains understandable without them. > Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. date — pass: originalPublished 2026-11-04 matches the v1 submission date on the arXiv page. It does not use the later feed date. > \[Submitted on 4 Nov 2026\] support — uncertain: The source-stated claims match the abstract verbatim. The labeled interpretation is hedged and reasonable: completing research steps and knowing physics may not suffice. However, its attached excerpt describes the shortcomings of existing benchmarks and does not support that inference. The supporting text is the authors' diagnosis that weaknesses lie in rigor and intuition rather than knowledge or execution. Smallest repair: re-pair the interpretation claim with the excerpt quoted here. > frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors. media — pass: The selected media is the verified direct link to the arXiv abstract page of this paper. It is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/5d50891be8bbaaa9711e85357b858075c7ac481e841007328c2bb05cada864a3/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-frontierphysics-evaluating-agents-for-end-to-end-5d50891b.json)