{"id":"c9eec7e50c80cb88f95b541c19a76c421c00c3bfcdaa22912a2a9c0051b1398a","title":"arXiv cs.AI: HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","argument":"Authors Zhi Hong, Qianlong Wang, and coauthors introduce HarnessFlow, a framework for optimizing executable harnesses that govern how large language model (LLM) agents interact with tools and an environment. They hold the model and task environment fixed, using Monte Carlo Tree Search (MCTS) to select candidate harnesses for refinement with archived code, execution feedback, and reusable mechanisms. The authors report results on six benchmarks spanning agentic tasks, coding, and web interaction. With 15 refinement attempts per benchmark, HarnessFlow achieved an average score of 69.48%: 10.89 percentage points above the initial harnesses and 3.95 above Meta-Harness. They also report lower average per-task execution costs than hand-engineered baselines. The paper argues that harness design, which the authors describe as largely manual, can be treated as structured search under a limited evaluation budget. As an Escape Meridian interpretation, the results offer evidence for this approach on the tested benchmarks, but do not establish its performance on other tasks or settings.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence provided is the arXiv abstract, which reports results on six benchmarks but does not include details of their setup or results for other tasks. The reported test scores and execution costs describe the evaluated benchmarks and comparisons; they do not establish performance in other settings.","speaker":{"name":"Hong, Zhi, Wang, Qianlong, Wei, Hongchen, Wu, Aidan, Ying, Daohong, Kong, Mingze, Liu, Bei, Luo, Chong","roleAtSource":"Article author"},"source":{"title":"HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.06254","publishedAt":"2026-11-04","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present HarnessFlow as a framework for optimizing complete executable harnesses while holding the model and task environment fixed.","excerpt":"We introduce HarnessFlow, a framework for optimizing complete executable harnesses under a limited candidate-evaluation budget, with the model and task environment held fixed.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"HarnessFlow uses an MCTS-based policy to choose harness candidates for refinement, using archived code, execution feedback, and reusable mechanisms.","excerpt":"using a Monte Carlo Tree Search (MCTS)-based policy to balance exploitation and exploration. A coding agent refines selected harnesses using archived code, execution feedback, and a library of reusable harness mechanisms.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The paper reports that HarnessFlow achieved the highest mean test scores among the compared methods on six benchmarks, with 15 refinement attempts per benchmark.","excerpt":"With 15 refinement attempts per benchmark, HarnessFlow achieves the highest mean test scores among the compared methods on six benchmarks spanning agentic tasks, coding, and web interaction.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The authors report an average score of 69.48%, exceeding initial harnesses by 10.89 percentage points and Meta-Harness by 3.95 percentage points.","excerpt":"The optimized harnesses achieve an average score of 69.48%, outperforming the initial harnesses and Meta-Harness by 10.89 and 3.95 percentage points, respectively","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The authors report lower average per-task execution costs than hand-engineered baselines.","excerpt":"while incurring lower average per-task execution costs than hand-engineered baselines.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The reported evaluation covers six benchmarks; whether the reported advantages extend to other tasks remains untested in this evidence.","excerpt":"on six benchmarks spanning agentic tasks, coding, and web interaction.","url":"https://arxiv.org/abs/2611.06254","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.06254","checkedAt":"2026-11-10T04:34:09.136Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would HarnessFlow’s reported score and cost advantages hold on tasks beyond the six benchmarks evaluated?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","summary":"Authors Zhi Hong, Qianlong Wang, and coauthors introduce HarnessFlow, a framework for optimizing executable harnesses that govern how large language model (LLM) agents interact with tools and an environment. They hold the model and task environment fixed, using Monte Carlo Tree Search (MCTS) to select candidate harnesses for refinement with archived code, execution feedback, and reusable mechanisms. The authors report results on six benchmarks spanning agentic tasks, coding, and web interaction. With 15 refinement attempts per benchmark, HarnessFlow achieved an average score of 69.48%: 10.89 percentage points above the initial harnesses and 3.95 above Meta-Harness. They also report lower average per-task execution costs than hand-engineered baselines. The paper argues that harness design, which the authors describe as largely manual, can be treated as structured search under a limited evaluation budget. As an Escape Meridian interpretation, the results offer evidence for this approach on the tested benchmarks, but do not establish its performance on other tasks or settings.","question":"Would HarnessFlow’s reported score and cost advantages hold on tasks beyond the six benchmarks evaluated?","claims":[{"text":"The authors present HarnessFlow as a framework for optimizing complete executable harnesses while holding the model and task environment fixed.","excerpt":"We introduce HarnessFlow, a framework for optimizing complete executable harnesses under a limited candidate-evaluation budget, with the model and task environment held fixed.","support":"source-stated"},{"text":"HarnessFlow uses an MCTS-based policy to choose harness candidates for refinement, using archived code, execution feedback, and reusable mechanisms.","excerpt":"using a Monte Carlo Tree Search (MCTS)-based policy to balance exploitation and exploration. A coding agent refines selected harnesses using archived code, execution feedback, and a library of reusable harness mechanisms.","support":"source-stated"},{"text":"The paper reports that HarnessFlow achieved the highest mean test scores among the compared methods on six benchmarks, with 15 refinement attempts per benchmark.","excerpt":"With 15 refinement attempts per benchmark, HarnessFlow achieves the highest mean test scores among the compared methods on six benchmarks spanning agentic tasks, coding, and web interaction.","support":"source-stated"},{"text":"The authors report an average score of 69.48%, exceeding initial harnesses by 10.89 percentage points and Meta-Harness by 3.95 percentage points.","excerpt":"The optimized harnesses achieve an average score of 69.48%, outperforming the initial harnesses and Meta-Harness by 10.89 and 3.95 percentage points, respectively","support":"source-stated"},{"text":"The authors report lower average per-task execution costs than hand-engineered baselines.","excerpt":"while incurring lower average per-task execution costs than hand-engineered baselines.","support":"source-stated"},{"text":"The reported evaluation covers six benchmarks; whether the reported advantages extend to other tasks remains untested in this evidence.","excerpt":"on six benchmarks spanning agentic tasks, coding, and web interaction.","support":"interpretation"}],"limitations":["The evidence provided is the arXiv abstract, which reports results on six benchmarks but does not include details of their setup or results for other tasks.","The reported test scores and execution costs describe the evaluated benchmarks and comparisons; they do not establish performance in other settings."],"mediaId":"22b058e98cf7c8abfbd21be536b60c8b4bb1d809937a25cebbd657bd4692541f","correctionReason":null,"id":"c9eec7e50c80cb88f95b541c19a76c421c00c3bfcdaa22912a2a9c0051b1398a","publicationId":"94dbfe9e-6959-4a73-8830-95b33ed4502a","version":1,"sourceUrl":"https://arxiv.org/abs/2611.06254","sourceTitle":"HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-04","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:31.868Z","createdAt":"2026-11-10T04:02:14.881Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"22b058e98cf7c8abfbd21be536b60c8b4bb1d809937a25cebbd657bd4692541f","kind":"article","url":"https://arxiv.org/abs/2611.06254","sourceUrl":"https://arxiv.org/abs/2611.06254","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T04:01:33.641Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.06254","checkedAt":"2026-11-10T04:34:09.136Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"22b058e98cf7c8abfbd21be536b60c8b4bb1d809937a25cebbd657bd4692541f","confidence":0.86,"provider":"typesafe","model":"jev-1.13.0","inputHash":"f21dc4e398eeb9d2b712ff2255d690c1d725126827840c82a43d7d8aefde827a","decidedAt":"2026-11-10T04:01:34.533Z"},"provenance":{"textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observationId":"9b940800b7a40066b79697074c5e3f6052adfdd13b9f768c83b5f9ee7682a362","retrievedAt":"2026-11-10T04:01:33.525Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"56ecc947-a589-4ed8-8c7e-1af1dd4e6baa"},"reviewer":{"model":"gpt-6-luna","runId":"74f04b8a-8344-4af0-8f8d-19d0b16e72e5"}},"audit":{"status":"passed","reviewedAt":"2026-11-10T04:02:14.881Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Hong, Zhi","url":null,"kind":"unknown"},{"name":"Wang, Qianlong","url":null,"kind":"unknown"},{"name":"Wei, Hongchen","url":null,"kind":"unknown"},{"name":"Wu, Aidan","url":null,"kind":"unknown"},{"name":"Ying, Daohong","url":null,"kind":"unknown"},{"name":"Kong, Mingze","url":null,"kind":"unknown"},{"name":"Liu, Bei","url":null,"kind":"unknown"},{"name":"Luo, Chong","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Hong, Zhi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hong, Zhi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Wang, Qianlong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Qianlong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Wei, Hongchen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wei, Hongchen","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Wu, Aidan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wu, Aidan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Ying, Daohong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ying, Daohong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Kong, Mingze","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Kong, Mingze","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Liu, Bei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Bei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Luo, Chong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luo, Chong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.06254","snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c9eec7e50c80cb88f95b541c19a76c421c00c3bfcdaa22912a2a9c0051b1398a/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-harnessflow-mcts-guided-harness-optimization-for-c9eec7e5","formats":{"html":"/readings/arxiv-cs-ai-harnessflow-mcts-guided-harness-optimization-for-c9eec7e5","markdown":"/readings/arxiv-cs-ai-harnessflow-mcts-guided-harness-optimization-for-c9eec7e5.md","json":"/readings/arxiv-cs-ai-harnessflow-mcts-guided-harness-optimization-for-c9eec7e5.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-agentic","name":"Agentic AI","explanation":"AI systems that plan and carry out a series of steps toward a goal, often by using other software, rather than giving one answer.","referenceUrl":"https://developers.google.com/machine-learning/glossary#agentic"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"c9eec7e50c80cb88f95b541c19a76c421c00c3bfcdaa22912a2a9c0051b1398a","revision":"94dbfe9e-6959-4a73-8830-95b33ed4502a","revisionKind":"publication-id","title":"arXiv cs.AI: HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","url":"https://escapemeridian.com/readings/arxiv-cs-ai-harnessflow-mcts-guided-harness-optimization-for-c9eec7e5"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.06254","revision":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","revisionKind":"sha256-source-text","title":"HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","url":"https://arxiv.org/abs/2611.06254"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"c9eec7e50c80cb88f95b541c19a76c421c00c3bfcdaa22912a2a9c0051b1398a","title":"arXiv cs.AI: HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","argument":"Authors Zhi Hong, Qianlong Wang, and coauthors introduce HarnessFlow, a framework for optimizing executable harnesses that govern how large language model (LLM) agents interact with tools and an environment. They hold the model and task environment fixed, using Monte Carlo Tree Search (MCTS) to select candidate harnesses for refinement with archived code, execution feedback, and reusable mechanisms. The authors report results on six benchmarks spanning agentic tasks, coding, and web interaction. With 15 refinement attempts per benchmark, HarnessFlow achieved an average score of 69.48%: 10.89 percentage points above the initial harnesses and 3.95 above Meta-Harness. They also report lower average per-task execution costs than hand-engineered baselines. The paper argues that harness design, which the authors describe as largely manual, can be treated as structured search under a limited evaluation budget. As an Escape Meridian interpretation, the results offer evidence for this approach on the tested benchmarks, but do not establish its performance on other tasks or settings.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The evidence provided is the arXiv abstract, which reports results on six benchmarks but does not include details of their setup or results for other tasks. The reported test scores and execution costs describe the evaluated benchmarks and comparisons; they do not establish performance in other settings.","speaker":{"name":"Hong, Zhi, Wang, Qianlong, Wei, Hongchen, Wu, Aidan, Ying, Daohong, Kong, Mingze, Liu, Bei, Luo, Chong","roleAtSource":"Article author"},"source":{"title":"HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.06254","publishedAt":"2026-11-04","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors present HarnessFlow as a framework for optimizing complete executable harnesses while holding the model and task environment fixed.","excerpt":"We introduce HarnessFlow, a framework for optimizing complete executable harnesses under a limited candidate-evaluation budget, with the model and task environment held fixed.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"HarnessFlow uses an MCTS-based policy to choose harness candidates for refinement, using archived code, execution feedback, and reusable mechanisms.","excerpt":"using a Monte Carlo Tree Search (MCTS)-based policy to balance exploitation and exploration. A coding agent refines selected harnesses using archived code, execution feedback, and a library of reusable harness mechanisms.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The paper reports that HarnessFlow achieved the highest mean test scores among the compared methods on six benchmarks, with 15 refinement attempts per benchmark.","excerpt":"With 15 refinement attempts per benchmark, HarnessFlow achieves the highest mean test scores among the compared methods on six benchmarks spanning agentic tasks, coding, and web interaction.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The authors report an average score of 69.48%, exceeding initial harnesses by 10.89 percentage points and Meta-Harness by 3.95 percentage points.","excerpt":"The optimized harnesses achieve an average score of 69.48%, outperforming the initial harnesses and Meta-Harness by 10.89 and 3.95 percentage points, respectively","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The authors report lower average per-task execution costs than hand-engineered baselines.","excerpt":"while incurring lower average per-task execution costs than hand-engineered baselines.","url":"https://arxiv.org/abs/2611.06254","relationship":"Source-stated"},{"text":"The reported evaluation covers six benchmarks; whether the reported advantages extend to other tasks remains untested in this evidence.","excerpt":"on six benchmarks spanning agentic tasks, coding, and web interaction.","url":"https://arxiv.org/abs/2611.06254","relationship":"Escape Meridian interpretation"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.06254","checkedAt":"2026-11-10T04:34:09.136Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Would HarnessFlow’s reported score and cost advantages hold on tasks beyond the six benchmarks evaluated?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","summary":"Authors Zhi Hong, Qianlong Wang, and coauthors introduce HarnessFlow, a framework for optimizing executable harnesses that govern how large language model (LLM) agents interact with tools and an environment. They hold the model and task environment fixed, using Monte Carlo Tree Search (MCTS) to select candidate harnesses for refinement with archived code, execution feedback, and reusable mechanisms. The authors report results on six benchmarks spanning agentic tasks, coding, and web interaction. With 15 refinement attempts per benchmark, HarnessFlow achieved an average score of 69.48%: 10.89 percentage points above the initial harnesses and 3.95 above Meta-Harness. They also report lower average per-task execution costs than hand-engineered baselines. The paper argues that harness design, which the authors describe as largely manual, can be treated as structured search under a limited evaluation budget. As an Escape Meridian interpretation, the results offer evidence for this approach on the tested benchmarks, but do not establish its performance on other tasks or settings.","question":"Would HarnessFlow’s reported score and cost advantages hold on tasks beyond the six benchmarks evaluated?","claims":[{"text":"The authors present HarnessFlow as a framework for optimizing complete executable harnesses while holding the model and task environment fixed.","excerpt":"We introduce HarnessFlow, a framework for optimizing complete executable harnesses under a limited candidate-evaluation budget, with the model and task environment held fixed.","support":"source-stated"},{"text":"HarnessFlow uses an MCTS-based policy to choose harness candidates for refinement, using archived code, execution feedback, and reusable mechanisms.","excerpt":"using a Monte Carlo Tree Search (MCTS)-based policy to balance exploitation and exploration. A coding agent refines selected harnesses using archived code, execution feedback, and a library of reusable harness mechanisms.","support":"source-stated"},{"text":"The paper reports that HarnessFlow achieved the highest mean test scores among the compared methods on six benchmarks, with 15 refinement attempts per benchmark.","excerpt":"With 15 refinement attempts per benchmark, HarnessFlow achieves the highest mean test scores among the compared methods on six benchmarks spanning agentic tasks, coding, and web interaction.","support":"source-stated"},{"text":"The authors report an average score of 69.48%, exceeding initial harnesses by 10.89 percentage points and Meta-Harness by 3.95 percentage points.","excerpt":"The optimized harnesses achieve an average score of 69.48%, outperforming the initial harnesses and Meta-Harness by 10.89 and 3.95 percentage points, respectively","support":"source-stated"},{"text":"The authors report lower average per-task execution costs than hand-engineered baselines.","excerpt":"while incurring lower average per-task execution costs than hand-engineered baselines.","support":"source-stated"},{"text":"The reported evaluation covers six benchmarks; whether the reported advantages extend to other tasks remains untested in this evidence.","excerpt":"on six benchmarks spanning agentic tasks, coding, and web interaction.","support":"interpretation"}],"limitations":["The evidence provided is the arXiv abstract, which reports results on six benchmarks but does not include details of their setup or results for other tasks.","The reported test scores and execution costs describe the evaluated benchmarks and comparisons; they do not establish performance in other settings."],"mediaId":"22b058e98cf7c8abfbd21be536b60c8b4bb1d809937a25cebbd657bd4692541f","correctionReason":null,"id":"c9eec7e50c80cb88f95b541c19a76c421c00c3bfcdaa22912a2a9c0051b1398a","publicationId":"94dbfe9e-6959-4a73-8830-95b33ed4502a","version":1,"sourceUrl":"https://arxiv.org/abs/2611.06254","sourceTitle":"HarnessFlow: MCTS-Guided Harness Optimization for LLM Agents","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-04","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-05T05:06:31.868Z","createdAt":"2026-11-10T04:02:14.881Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"22b058e98cf7c8abfbd21be536b60c8b4bb1d809937a25cebbd657bd4692541f","kind":"article","url":"https://arxiv.org/abs/2611.06254","sourceUrl":"https://arxiv.org/abs/2611.06254","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-10T04:01:33.641Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.06254","checkedAt":"2026-11-10T04:34:09.136Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"22b058e98cf7c8abfbd21be536b60c8b4bb1d809937a25cebbd657bd4692541f","confidence":0.86,"provider":"typesafe","model":"jev-1.13.0","inputHash":"f21dc4e398eeb9d2b712ff2255d690c1d725126827840c82a43d7d8aefde827a","decidedAt":"2026-11-10T04:01:34.533Z"},"provenance":{"textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observationId":"9b940800b7a40066b79697074c5e3f6052adfdd13b9f768c83b5f9ee7682a362","retrievedAt":"2026-11-10T04:01:33.525Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"56ecc947-a589-4ed8-8c7e-1af1dd4e6baa"},"reviewer":{"model":"gpt-6-luna","runId":"74f04b8a-8344-4af0-8f8d-19d0b16e72e5"}},"audit":{"status":"passed","reviewedAt":"2026-11-10T04:02:14.881Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Hong, Zhi","url":null,"kind":"unknown"},{"name":"Wang, Qianlong","url":null,"kind":"unknown"},{"name":"Wei, Hongchen","url":null,"kind":"unknown"},{"name":"Wu, Aidan","url":null,"kind":"unknown"},{"name":"Ying, Daohong","url":null,"kind":"unknown"},{"name":"Kong, Mingze","url":null,"kind":"unknown"},{"name":"Liu, Bei","url":null,"kind":"unknown"},{"name":"Luo, Chong","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Hong, Zhi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hong, Zhi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Wang, Qianlong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wang, Qianlong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Wei, Hongchen","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wei, Hongchen","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Wu, Aidan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wu, Aidan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Ying, Daohong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ying, Daohong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Kong, Mingze","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Kong, Mingze","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Liu, Bei","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Liu, Bei","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}},{"actor":{"name":"Luo, Chong","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luo, Chong","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.06254","locator":"head/meta[name=author|citation_author]"},"assertionId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T04:34:09.136Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","sourceUrl":"https://arxiv.org/abs/2611.06254","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.06254","snapshotId":"e321bfaa1288dab9720f65cbde5ad4dbc680336b83c2ab6523f882b605db428c","contentSha256":"2f691a57dfcbebe9f5c890660d24794d1666bfb3fef40b167f26eb2d905ed84a","textHash":"3d326fe672c767a688e9470a0915f103d2263836070c1da2e783430cab2d2003","observedAt":"2026-11-10T04:34:09.136Z","basis":"publisher-page-metadata","sourceLinks":46,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c9eec7e50c80cb88f95b541c19a76c421c00c3bfcdaa22912a2a9c0051b1398a/source"}}}],"audits":[{"publicationId":"94dbfe9e-6959-4a73-8830-95b33ed4502a","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft correctly attributes HarnessFlow and its results to the paper's authors, naming the first two listed authors plus coauthors. Reported scores and costs are framed as author reports, and the generalization caveat is labeled as Escape Meridian interpretation.","excerpt":"Authors:Zhi Hong, Qianlong Wang, Hongchen Wei, Aidan Wu, Daohong Ying, Mingze Kong, Bei Liu, Chong Luo"},{"dimension":"summary","status":"pass","explanation":"The summary leads with the development, gives a plain description of a harness (what governs how agents interact with tools and the environment), and keeps the fixed model and environment, the search mechanism, the budget and the results. MCTS and Meta-Harness get little explanation, but the reading remains understandable, and term tooltips can cover them. The title mirrors the source, and the question is a genuine open issue.","excerpt":"The performance of large language model (LLM) agents depends on both the underlying model and the harness that governs interaction with tools and the environment."},{"dimension":"date","status":"pass","explanation":"originalPublished 2026-11-04 matches the arXiv v1 submission date. It is not the later feed date of 2026-11-05.","excerpt":"[Submitted on 4 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"All numeric claims match the abstract: 15 attempts, six benchmarks, 69.48%, +10.89 and +3.95 points. The cost claim is correctly scoped to hand-engineered baselines. The phrase 'can be treated as structured search' is slightly firmer than the authors' 'promising route', but it is not a material change. The interpretation and limitations do not overstate the evidence.","excerpt":"These results suggest that structured harness search offers a promising route toward automating the design of capable and efficient LLM agents."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original arXiv abstract page, which is the most relevant way to reach the source.","excerpt":null}],"createdAt":"2026-11-10T04:02:14.881Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}