You, Bingran, Ke, Wenjun, Rosenauer, Ria, Liu, Bo, Chen, Wenbo, Chen, Min, Xu, Shuqi, Ma, Yue, Feng, Jiechao, Chen, Bo, Sun, Xiangkai, Liu, Jiarui, Guo, Jinen, Zhou, Zeyu, Huang, Xu, Li, Zhuohan, Li, Haoyu, Lin, Heguang, Ding, Lijie, Zeng, Yadong, Zhang, Kangqi, Liu, Xinge, Liu, Hanwen, Li, Zhenghong, von Recum, Alexander, Zhu, Yuchen, Schwarzfischer, Felix, Zhao, Yizhou, Wang, Jicheng, Li, Binxu, Yu, Qian, Li, Chan, Dai, Cheng, Wang, Di, Zhang, Haoyu, Sun, Jiankai, Feng, Kaituo, Qin, Mian, Yin, Ruihong, Luo, Yiqun, Wu, Weiwei, Pan, Shihong, Hsieh, Tsung Yeh, Huang, Xiao, Zhang, Yan, Li, Sihan, Ren, Zirui, Tu, Zengji, Choe, Kyoung Whan, Sun, Ke, Galia, Vincenzo, Mishra, Vaisakhi, He, Jiashu, Wang, Hongyang, Yang, Zhiheng, Gaur, Shubham, Gu, Yifan, Wang, Shuyi, Zuo, Zihao, Dillmann, Steven, Sun, Yiyou, Han, Xinyang, Lee, Hanchung, Tao, Molei, Zhao, Yue, Liu, Shilong, Wu, Yuchen, Zhao, Xuandong, Li, Xiangyi, Brand, Florian, Ceder, Gerbrand, Haeffner, Hartmut · Nov 4, 2026
arXiv cs.AI: FrontierPhysics: Evaluating Agents for End-to-End Physics Research
Markdown · JSON · Original source
The authors introduce FrontierPhysics, a designed to test across the stages of physics research rather than on fixed questions or isolated tasks. Its 56 tasks, created by 42 physicists from their own research, span nine domains and require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts. The authors combine deterministic verifiers with expert-authored rubrics to assess correctness and whether results are publishable. Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts. The authors say manual review of agent trajectories points to insufficiently rigorous reasoning and limited expert intuition—not gaps in physics knowledge or execution errors—as the main weaknesses of frontier agents. Escape Meridian's reading is that this distinction matters: completing research steps and knowing physics may not be enough to meet the benchmark’s standard for research quality. The abstract does not establish how performance on these tasks translates to work in physics research beyond the benchmark.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T11:26:06.775Z. First observed: 2026-11-05T05:00:33.388Z. Retrieved: 2026-11-09T11:25:17.265Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The evidence supplied is the abstract and page text, not the full paper; it does not provide the rubric details or explain how publishability is judged. The abstract reports results but does not establish how they translate to physics research beyond the benchmark.
Open question
How do the authors determine, through their expert-authored rubrics, whether an agent’s research is publishable?
Source-stated
FrontierPhysics evaluates agents on 56 research tasks created by 42 physicists, covering nine domains and requiring multiple stages of research.
an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains
Source-stated
Tasks require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts.
Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts.
Source-stated
The benchmark combines deterministic verifiers and expert-authored rubrics to evaluate correctness and publishability.
Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable.
Source-stated
Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts.
Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts.
Source-stated
The authors identify insufficiently rigorous reasoning and limited expert intuition as frontier agents’ main weaknesses, rather than physics knowledge gaps or execution errors.
frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
Escape Meridian interpretation
The benchmark results suggest that completing research steps and having physics knowledge may not suffice to meet this benchmark’s standard for research quality.
existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness.
Source and contributor context
You, Bingran, Ke, Wenjun, Rosenauer, Ria, Liu, Bo, Chen, Wenbo, Chen, Min, Xu, Shuqi, Ma, Yue, Feng, Jiechao, Chen, Bo, Sun, Xiangkai, Liu, Jiarui, Guo, Jinen, Zhou, Zeyu, Huang, Xu, Li, Zhuohan, Li, Haoyu, Lin, Heguang, Ding, Lijie, Zeng, Yadong, Zhang, Kangqi, Liu, Xinge, Liu, Hanwen, Li, Zhenghong, von Recum, Alexander, Zhu, Yuchen, Schwarzfischer, Felix, Zhao, Yizhou, Wang, Jicheng, Li, Binxu, Yu, Qian, Li, Chan, Dai, Cheng, Wang, Di, Zhang, Haoyu, Sun, Jiankai, Feng, Kaituo, Qin, Mian, Yin, Ruihong, Luo, Yiqun, Wu, Weiwei, Pan, Shihong, Hsieh, Tsung Yeh, Huang, Xiao, Zhang, Yan, Li, Sihan, Ren, Zirui, Tu, Zengji, Choe, Kyoung Whan, Sun, Ke, Galia, Vincenzo, Mishra, Vaisakhi, He, Jiashu, Wang, Hongyang, Yang, Zhiheng, Gaur, Shubham, Gu, Yifan, Wang, Shuyi, Zuo, Zihao, Dillmann, Steven, Sun, Yiyou, Han, Xinyang, Lee, Hanchung, Tao, Molei, Zhao, Yue, Liu, Shilong, Wu, Yuchen, Zhao, Xuandong, Li, Xiangyi, Brand, Florian, Ceder, Gerbrand, Haeffner, Hartmut · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
AI agent
An AI system that chooses and carries out actions toward a goal, often using software tools.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 5d943385-64cc-4b79-844e-a1923038164e.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:26:06.775Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 2972272e69ed0df6350dbfc1e93e0d43368cd4dd18279899684218a411d66829. Source text hash: 4cd1efe5bad459af3ec03bf9b3f487d3ceea3d737fe235fcb6429853930b4291.
Revision and source-check history
Version 1 · 2026-11-09T11:26:06.775Z
arXiv cs.AI: FrontierPhysics: Evaluating Agents for End-to-End Physics Research
The authors introduce FrontierPhysics, a designed to test across the stages of physics research rather than on fixed questions or isolated tasks. Its 56 tasks, created by 42 physicists from their own research, span nine domains and require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts. The authors combine deterministic verifiers with expert-authored rubrics to assess correctness and whether results are publishable. Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts. The authors say manual review of agent trajectories points to insufficiently rigorous reasoning and limited expert intuition—not gaps in physics knowledge or execution errors—as the main weaknesses of frontier agents. Escape Meridian's reading is that this distinction matters: completing research steps and knowing physics may not be enough to meet the benchmark’s standard for research quality. The abstract does not establish how performance on these tasks translates to work in physics research beyond the benchmark.
Original source
cs.AI · Article
Source details
Original source date: Nov 4, 2026 (2026-11-04).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-09T11:26:06.775Z. First observed: 2026-11-05T05:00:33.388Z. Retrieved: 2026-11-09T11:25:17.265Z.
Limits
The evidence supplied is the abstract and page text, not the full paper; it does not provide the rubric details or explain how publishability is judged. The abstract reports results but does not establish how they translate to physics research beyond the benchmark.
Open question
How do the authors determine, through their expert-authored rubrics, whether an agent’s research is publishable?
Source-stated
FrontierPhysics evaluates agents on 56 research tasks created by 42 physicists, covering nine domains and requiring multiple stages of research.
an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains
Source-stated
Tasks require agents to review literature, plan and conduct research and experiments, and produce structured outputs and scientific manuscripts.
Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts.
Source-stated
The benchmark combines deterministic verifiers and expert-authored rubrics to evaluate correctness and publishability.
Combining deterministic verifiers with expert-authored rubrics, FrontierPhysics evaluates whether agents' research is correct and publishable.
Source-stated
Across 22 models in 26 configurations, no configuration passed more than 46 of 168 rollouts.
Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts.
Source-stated
The authors identify insufficiently rigorous reasoning and limited expert intuition as frontier agents’ main weaknesses, rather than physics knowledge gaps or execution errors.
frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
Escape Meridian interpretation
The benchmark results suggest that completing research steps and having physics knowledge may not suffice to meet this benchmark’s standard for research quality.
existing physics benchmarks only evaluate fixed questions, isolated sub-tasks, or paper reproduction with given solutions, and rarely check research quality alongside correctness.
Source and contributor context
You, Bingran, Ke, Wenjun, Rosenauer, Ria, Liu, Bo, Chen, Wenbo, Chen, Min, Xu, Shuqi, Ma, Yue, Feng, Jiechao, Chen, Bo, Sun, Xiangkai, Liu, Jiarui, Guo, Jinen, Zhou, Zeyu, Huang, Xu, Li, Zhuohan, Li, Haoyu, Lin, Heguang, Ding, Lijie, Zeng, Yadong, Zhang, Kangqi, Liu, Xinge, Liu, Hanwen, Li, Zhenghong, von Recum, Alexander, Zhu, Yuchen, Schwarzfischer, Felix, Zhao, Yizhou, Wang, Jicheng, Li, Binxu, Yu, Qian, Li, Chan, Dai, Cheng, Wang, Di, Zhang, Haoyu, Sun, Jiankai, Feng, Kaituo, Qin, Mian, Yin, Ruihong, Luo, Yiqun, Wu, Weiwei, Pan, Shihong, Hsieh, Tsung Yeh, Huang, Xiao, Zhang, Yan, Li, Sihan, Ren, Zirui, Tu, Zengji, Choe, Kyoung Whan, Sun, Ke, Galia, Vincenzo, Mishra, Vaisakhi, He, Jiashu, Wang, Hongyang, Yang, Zhiheng, Gaur, Shubham, Gu, Yifan, Wang, Shuyi, Zuo, Zihao, Dillmann, Steven, Sun, Yiyou, Han, Xinyang, Lee, Hanchung, Tao, Molei, Zhao, Yue, Liu, Shilong, Wu, Yuchen, Zhao, Xuandong, Li, Xiangyi, Brand, Florian, Ceder, Gerbrand, Haeffner, Hartmut · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
AI agent
An AI system that chooses and carries out actions toward a goal, often using software tools.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 5d943385-64cc-4b79-844e-a1923038164e.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:26:06.775Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 2972272e69ed0df6350dbfc1e93e0d43368cd4dd18279899684218a411d66829. Source text hash: 4cd1efe5bad459af3ec03bf9b3f487d3ceea3d737fe235fcb6429853930b4291.
uncertain · 2026-11-09T11:26:06.775Z · anthropic/claude-opus-5.5
Publication: 5d943385-64cc-4b79-844e-a1923038164e
attribution — pass: The reading is attributed to the arXiv cs.AI paper and its authors. Benchmark design, results and the diagnosis of agent weaknesses are framed as what the authors introduce or say. The Escape Meridian inference is explicitly labeled as EM's reading.
We introduce FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research
summary — pass: The summary leads with what was introduced and why: existing benchmarks test fixed questions or isolated sub-tasks. It accurately conveys task scope, evaluation method, headline result and diagnosed weaknesses. The question is a genuine open issue the abstract leaves unresolved. Terms such as rollouts, configurations, deterministic verifiers and agent trajectories are kept exactly as the source uses them. They could use term-library tooltips; for example, rollout means an attempt. The central finding remains understandable without them.
Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts.
date — pass: originalPublished 2026-11-04 matches the v1 submission date on the arXiv page. It does not use the later feed date.
[Submitted on 4 Nov 2026]
support — uncertain: The source-stated claims match the abstract verbatim. The labeled interpretation is hedged and reasonable: completing research steps and knowing physics may not suffice. However, its attached excerpt describes the shortcomings of existing benchmarks and does not support that inference. The supporting text is the authors' diagnosis that weaknesses lie in rigor and intuition rather than knowledge or execution. Smallest repair: re-pair the interpretation claim with the excerpt quoted here.
frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors.
media — pass: The selected media is the verified direct link to the arXiv abstract page of this paper. It is the most direct access to the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceAn AI system that chooses and carries out actions toward a goal, often using software tools.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceAn AI system that chooses and carries out actions toward a goal, often using software tools.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference