Arvind Narayanan and Sayash Kapoor · Aug 5, 2026
Sayash Kapoor and Arvind Narayanan argue AI agents struggle with open-ended research
Markdown · JSON · Original source
The essay was published on August 5, 2026. Sayash Kapoor and Arvind Narayanan report that agents failed to produce papers accepted by the original authors in two shadow evaluations, where agents attempted to answer research questions from unpublished AI papers. Their analysis found problems with judgment, use of available resources, response to feedback, backtracking, and following instructions. The authors distinguish these open-ended tasks, where success may not be immediately verifiable, from narrow tasks with clear measures of success. The authors say the results are tentative and the evaluation has important limits: it covers only two papers, and expert reviewers knew the papers were AI-generated. They argue that the findings raise a larger question about whether progress on verifiable tasks can drive broad , or whether open-ended research and other bottlenecks will constrain the pace of AI progress. Their conclusion is that identifying which bottlenecks are difficult to overcome could substantially affect estimates of that pace. They plan further shadow evaluations; whether open-ended research is on the critical path to recursive self-improvement remains unresolved.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
Arvind Narayanan and Sayash Kapoor · Article
Source details
Original source date: Aug 5, 2026 (2026-08-05T13:49:19.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-06T06:33:24.661Z. First observed: 2026-10-23T20:55:41.490Z. Retrieved: 2026-10-23T20:57:35.496Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Text as retrieved on October 23, 2026; originally published August 5, 2026.
Open question
Will further shadow evaluations show that poor performance on open-ended AI research is a bottleneck on the critical path to ?
Source-stated
Sayash Kapoor and Arvind Narayanan report that authors rejected both papers produced by agents in their shadow evaluations.
The authors unambiguously rejected both agent papers.
Source-stated
The evaluations involved two papers, and the authors identify the small sample as a limitation.
Because we are conducting in-depth evaluations of each paper, the sample size is small (in our study, we used just two papers).
Source-stated
The authors say their results are tentative and intend to continue shadow evaluations.
Still, these findings are tentative, and we are working to address the limitations, such as by increasing the sample size, testing with new models, and through potential scaffold improvements.
Source-stated
The authors argue that whether further bottlenecks emerge and how tractable they are matters for understanding the pace of AI progress.
Whether we encounter further bottlenecks, and how tractable they turn out to be, will be consequential for understanding the pace of progress.
Source and contributor context
Arvind Narayanan and Sayash Kapoor · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
AI agent
An AI system that chooses and carries out actions toward a goal, often using software tools.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
Recursive self-improvement
An AI system improving its own capabilities, such as its code, training methods or reasoning, then using the gains to improve further. Each cycle could hypothetically produce a more capable system.
Editorial provenance
Version 1. Publication ID: b22eb2b3-af24-aac4-99fc-a6d494c79b2e.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-06T06:28:36.664Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: a2bee9e0eec4dbff5928f2c91dc2aab201ba2d5cc5ae2bf99205fbfd17781158. Source text hash: f5656aa798e0ac05d1793e9efd24070c525d045b1a6ae211c248d371c0294fdd.
Revision and source-check history
Version 1 · 2026-11-06T06:33:24.661Z
Sayash Kapoor and Arvind Narayanan argue AI agents struggle with open-ended research
The essay was published on August 5, 2026. Sayash Kapoor and Arvind Narayanan report that agents failed to produce papers accepted by the original authors in two shadow evaluations, where agents attempted to answer research questions from unpublished AI papers. Their analysis found problems with judgment, use of available resources, response to feedback, backtracking, and following instructions. The authors distinguish these open-ended tasks, where success may not be immediately verifiable, from narrow tasks with clear measures of success. The authors say the results are tentative and the evaluation has important limits: it covers only two papers, and expert reviewers knew the papers were AI-generated. They argue that the findings raise a larger question about whether progress on verifiable tasks can drive broad , or whether open-ended research and other bottlenecks will constrain the pace of AI progress. Their conclusion is that identifying which bottlenecks are difficult to overcome could substantially affect estimates of that pace. They plan further shadow evaluations; whether open-ended research is on the critical path to recursive self-improvement remains unresolved.
Original source
Arvind Narayanan and Sayash Kapoor · Article
Source details
Original source date: Aug 5, 2026 (2026-08-05T13:49:19.000Z).
Date basis: Date supplied by the publisher’s feed.
Added to Escape Meridian: 2026-11-06T06:33:24.661Z. First observed: 2026-10-23T20:55:41.490Z. Retrieved: 2026-10-23T20:57:35.496Z.
Limits
Text as retrieved on October 23, 2026; originally published August 5, 2026.
Open question
Will further shadow evaluations show that poor performance on open-ended AI research is a bottleneck on the critical path to ?
Source-stated
Sayash Kapoor and Arvind Narayanan report that authors rejected both papers produced by agents in their shadow evaluations.
The authors unambiguously rejected both agent papers.
Source-stated
The evaluations involved two papers, and the authors identify the small sample as a limitation.
Because we are conducting in-depth evaluations of each paper, the sample size is small (in our study, we used just two papers).
Source-stated
The authors say their results are tentative and intend to continue shadow evaluations.
Still, these findings are tentative, and we are working to address the limitations, such as by increasing the sample size, testing with new models, and through potential scaffold improvements.
Source-stated
The authors argue that whether further bottlenecks emerge and how tractable they are matters for understanding the pace of AI progress.
Whether we encounter further bottlenecks, and how tractable they turn out to be, will be consequential for understanding the pace of progress.
Source and contributor context
Arvind Narayanan and Sayash Kapoor · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
AI agent
An AI system that chooses and carries out actions toward a goal, often using software tools.
Frontier model
One of the most capable AI models available at a given time; the set shifts as models improve.
Recursive self-improvement
An AI system improving its own capabilities, such as its code, training methods or reasoning, then using the gains to improve further. Each cycle could hypothetically produce a more capable system.
Editorial provenance
Version 1. Publication ID: b22eb2b3-af24-aac4-99fc-a6d494c79b2e.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-06T06:28:36.664Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model.
Observation: a2bee9e0eec4dbff5928f2c91dc2aab201ba2d5cc5ae2bf99205fbfd17781158. Source text hash: f5656aa798e0ac05d1793e9efd24070c525d045b1a6ae211c248d371c0294fdd.
passed · 2026-11-06T06:28:36.664Z · anthropic/claude-opus-5.5
Publication: b22eb2b3-af24-aac4-99fc-a6d494c79b2e
attribution — pass: The publisher page is a single essay bylined Sayash Kapoor and Arvind Narayanan. The reading attributes the findings and arguments to them. The underlying paper has a longer author list, but the essay voice is theirs, so the attribution is accurate.
Sayash Kapoor and Arvind Narayanan Aug 05, 2026
summary — pass: The reading leads with the actors and the finding that both agent papers were rejected. It explains shadow evaluations inline and lists the five failure areas. It keeps the verifiable versus open-ended distinction, the stated limits, and the bottleneck argument. Recursive self-improvement, AI agent and frontier AI are defined beside the reading. The question follows from the source's stated open issue.
though it remains to be seen if it is on the critical path to RSI
date — pass: originalPublished is 2026-08-05T13:49:19Z, which matches the dateline Aug 05, 2026, and the reading states August 5, 2026. modifiedAt equals the publication time. The reading uses no hindsight framing, and planned evaluations are reported as plans rather than outcomes. This is a publisher page, so no archive disclosure is needed.
Aug 05, 2026
support — pass: The source text supports every claim: both papers rejected, two-paper sample, tentative results, plans for further shadow evaluations, and the importance of bottlenecks. The reading adds no unsupported benefits or costs.
We plan to continue shadow evaluations on a regular basis to help answer this question.
media — pass: The reading selects no media, and no verified media candidates were supplied, so selecting none is appropriate.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
One of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceAn AI system improving its own capabilities, such as its code, training methods or reasoning, then using the gains to improve further. Each cycle could hypothetically produce a more capable system.
Definition referenceAn AI system improving its own capabilities, such as its code, training methods or reasoning, then using the gains to improve further. Each cycle could hypothetically produce a more capable system.
Definition referenceOne of the most capable AI models available at a given time; the set shifts as models improve.
Definition referenceAn AI system improving its own capabilities, such as its code, training methods or reasoning, then using the gains to improve further. Each cycle could hypothetically produce a more capable system.
Definition referenceAn AI system improving its own capabilities, such as its code, training methods or reasoning, then using the gains to improve further. Each cycle could hypothetically produce a more capable system.
Definition reference