arXiv cs.AI: DARE links repair-task difficulty estimates to task-specific support

arXiv cs.AI: DARE links repair-task difficulty estimates to task-specific support

The authors of DARE argue that coding agents’ solve rates alone hide how difficult their solved tasks are and do not account for differences among benchmarks. Their approach measures difficulty as the observed failure rate across multiple agents, predicts it from 151 static features, and examines which of seven Repair-Task Feature Groups matter most for each task. It then uses those task-specific patterns to select support for the agent, rather than relying on a difficulty score alone. Across four benchmark partitions containing 2,594 tasks, predicted difficulty aligned with observed difficulty at Spearman’s ρ = .476–.621 on held-out partitions. In experiments on nine high-difficulty tasks, matching support to each task’s most salient Feature Group raised repair success from 54.7% to 79.6%. The authors present this as a way to improve automated program repair (APR) and compare agent performance across tasks of differing difficulty. The reported success increase is for those nine tasks; the abstract does not establish how broadly it carries over. The work also makes task difficulty part of evaluation, not just a score for ranking agents.

Publication: e5987b84-35ca-4ed4-86a3-bf423f1cf7ec

attribution — pass: The reading attributes the framework, the critique of solve rates, and all results to the paper's authors. The publisher label arXiv cs.AI matches the metadata. The author names (Lee, Jung, Nam) are not given, but nothing is misattributed.

Authors:Jikwang Lee, Seungwook Jung, Jaechang Nam

summary — uncertain: The core argument, method and results are faithfully conveyed. However, Spearman's ρ, the central accuracy measure, is left unexplained, so a general reader cannot judge what .476–.621 means. Smallest repair: add a term-library tooltip saying it measures how well the predicted ordering of task difficulty matches the observed ordering. The phrase 'predicted difficulty aligned with observed difficulty' also reinterprets the source's term 'aligned predictions'; keep the exact term. 'Support' and 'static features' would also benefit from tooltips. The question is substantive and follows from the evidence.

The difficulty model's aligned predictions achieve Spearman's $\rho$ = .476-.621 on held-out partitions

date — pass: The originalPublished date of 2026-11-04 matches the v1 submission. The 5 Nov date is a later revision (v2) and the feed date, not the first publication.

[Submitted on 4 Nov 2026 (v1), last revised 5 Nov 2026 (this version, v2)]

support — uncertain: The listed claims and the nine-task caveat are well supported. The closing sentence, 'The work also makes task difficulty part of evaluation, not just a score for ranking agents,' goes beyond the abstract. The authors say the findings provide 'a basis for' difficulty-stratified evaluation; they do not say the work establishes it. Smallest repair: 'The authors say it could provide a basis for difficulty-stratified agent evaluation and cross-benchmark comparison.'

while also providing a basis for difficulty-stratified agent evaluation, cross-benchmark comparisons

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.