{"id":"c95b2d566ae4f0801c86219a9c2a8d0e282ed0366f6960b5c9481d4563d2ab34","title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","argument":"The authors of Ask2Decide argue that LLM agents still struggle to ask people for information that changes a plan. Their benchmark tests 43 planning tasks: an operator holds facts missing from the request and records, while optimization solvers identify which facts change the optimal plan. Similar-sounding facts that do not change the plan test whether agents can distinguish useful information from irrelevant details. In 79 interaction sessions with 19 participants, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user: 43% versus 7%. The authors use those interactions to calibrate a simulated operator. Across four frontier LLMs, initial plans were infeasible on 58–98% of tasks when information was missing, versus 5–28% when all facts were supplied upfront. Agents uncovered facts that changed the optimal plan only slightly more often than facts that left it unchanged. The authors argue that this makes obtaining decision-relevant information through interaction a major bottleneck for LLM planning. The paper describes the work as ongoing and says it will be updated.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The reported evaluation covers 43 planning tasks, 79 interaction sessions with 19 participants, and four frontier LLMs; the abstract does not establish how well the results generalize beyond those settings. The authors describe the work as ongoing and say it will be updated.","speaker":{"name":"Tang, Yuhan, Fan, Ke, Zou, Xinkai, Mo, Baichuan, Cui, Kangxin, Qu, Ao, Zheng, Ou, Yu, Jiangbo, Koutsopoulos, Haris N., Zhao, Jinhua, Zhou, Yuan","roleAtSource":"Article author"},"source":{"title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.13015","publishedAt":"2026-11-09","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce Ask2Decide, a benchmark testing whether agents ask for missing information that affects planning decisions.","excerpt":"We introduce Ask2Decide, a benchmark of 43 planning tasks that evaluates whether agents ask for missing information that matters for their decisions.","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"In the reported participant sessions, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user.","excerpt":"human replies omit the correcting fact far more often than replies from an uncalibrated simulated user (43% versus 7%).","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"Across four evaluated frontier LLMs, initial plans were more often infeasible when information was missing than when all facts were provided upfront.","excerpt":"initial plans are infeasible on 58-98% of tasks when information is missing, compared with 5-28% when all facts are provided upfront.","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"The authors report that agents only slightly more often uncovered facts that changed the optimal plan than facts that did not change it.","excerpt":"Agents uncover facts that change the optimal plan only slightly more often than facts that leave it unchanged (39-52% versus 32-41%).","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"The authors characterize obtaining decision-relevant information through interaction as a major bottleneck for LLM planning.","excerpt":"These results show that obtaining decision-relevant information through interaction remains a major bottleneck for LLM planning.","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.13015","checkedAt":"2026-11-12T02:24:13.234Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"As this ongoing work is updated, do agents become better at uncovering facts that change the optimal plan than facts that leave it unchanged?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","summary":"The authors of Ask2Decide argue that LLM agents still struggle to ask people for information that changes a plan. Their benchmark tests 43 planning tasks: an operator holds facts missing from the request and records, while optimization solvers identify which facts change the optimal plan. Similar-sounding facts that do not change the plan test whether agents can distinguish useful information from irrelevant details. In 79 interaction sessions with 19 participants, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user: 43% versus 7%. The authors use those interactions to calibrate a simulated operator. Across four frontier LLMs, initial plans were infeasible on 58–98% of tasks when information was missing, versus 5–28% when all facts were supplied upfront. Agents uncovered facts that changed the optimal plan only slightly more often than facts that left it unchanged. The authors argue that this makes obtaining decision-relevant information through interaction a major bottleneck for LLM planning. The paper describes the work as ongoing and says it will be updated.","question":"As this ongoing work is updated, do agents become better at uncovering facts that change the optimal plan than facts that leave it unchanged?","claims":[{"text":"The authors introduce Ask2Decide, a benchmark testing whether agents ask for missing information that affects planning decisions.","excerpt":"We introduce Ask2Decide, a benchmark of 43 planning tasks that evaluates whether agents ask for missing information that matters for their decisions.","support":"source-stated"},{"text":"In the reported participant sessions, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user.","excerpt":"human replies omit the correcting fact far more often than replies from an uncalibrated simulated user (43% versus 7%).","support":"source-stated"},{"text":"Across four evaluated frontier LLMs, initial plans were more often infeasible when information was missing than when all facts were provided upfront.","excerpt":"initial plans are infeasible on 58-98% of tasks when information is missing, compared with 5-28% when all facts are provided upfront.","support":"source-stated"},{"text":"The authors report that agents only slightly more often uncovered facts that changed the optimal plan than facts that did not change it.","excerpt":"Agents uncover facts that change the optimal plan only slightly more often than facts that leave it unchanged (39-52% versus 32-41%).","support":"source-stated"},{"text":"The authors characterize obtaining decision-relevant information through interaction as a major bottleneck for LLM planning.","excerpt":"These results show that obtaining decision-relevant information through interaction remains a major bottleneck for LLM planning.","support":"source-stated"}],"limitations":["The reported evaluation covers 43 planning tasks, 79 interaction sessions with 19 participants, and four frontier LLMs; the abstract does not establish how well the results generalize beyond those settings.","The authors describe the work as ongoing and say it will be updated."],"mediaId":"590f86c72f2f70d65b2261b1b666cd78848a91f8aa6f740da5f588aa8bd76099","correctionReason":null,"id":"c95b2d566ae4f0801c86219a9c2a8d0e282ed0366f6960b5c9481d4563d2ab34","publicationId":"4fb012fa-ee07-4489-8357-b1d80de3b3e3","version":1,"sourceUrl":"https://arxiv.org/abs/2611.13015","sourceTitle":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-09","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-11T05:06:49.684Z","createdAt":"2026-11-12T00:17:45.770Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"590f86c72f2f70d65b2261b1b666cd78848a91f8aa6f740da5f588aa8bd76099","kind":"article","url":"https://arxiv.org/abs/2611.13015","sourceUrl":"https://arxiv.org/abs/2611.13015","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T00:16:42.607Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.13015","checkedAt":"2026-11-12T02:24:13.234Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"590f86c72f2f70d65b2261b1b666cd78848a91f8aa6f740da5f588aa8bd76099","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"b2079236ef484c7f3ca41e381e7e4196ba08d718b3db2cb50210a2e01f154746","decidedAt":"2026-11-12T00:16:43.574Z"},"provenance":{"textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observationId":"b24f2a1ea101f5ac270fc98f1550fb0f0a4313b3f94275e9543cd2ca4beefec5","retrievedAt":"2026-11-12T00:16:42.470Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"0ac6a20a-7164-4da7-8ad9-c45967018258"},"reviewer":{"model":"gpt-6-luna","runId":"29637b41-0d91-4826-81ea-f0eae7c7ac0e"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-12T00:17:45.770Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Tang, Yuhan","url":null,"kind":"unknown"},{"name":"Fan, Ke","url":null,"kind":"unknown"},{"name":"Zou, Xinkai","url":null,"kind":"unknown"},{"name":"Mo, Baichuan","url":null,"kind":"unknown"},{"name":"Cui, Kangxin","url":null,"kind":"unknown"},{"name":"Qu, Ao","url":null,"kind":"unknown"},{"name":"Zheng, Ou","url":null,"kind":"unknown"},{"name":"Yu, Jiangbo","url":null,"kind":"unknown"},{"name":"Koutsopoulos, Haris N.","url":null,"kind":"unknown"},{"name":"Zhao, Jinhua","url":null,"kind":"unknown"},{"name":"Zhou, Yuan","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Tang, Yuhan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tang, Yuhan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Fan, Ke","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Fan, Ke","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zou, Xinkai","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zou, Xinkai","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Mo, Baichuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Mo, Baichuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Cui, Kangxin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cui, Kangxin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Qu, Ao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qu, Ao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zheng, Ou","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zheng, Ou","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Yu, Jiangbo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yu, Jiangbo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Koutsopoulos, Haris N.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Koutsopoulos, Haris N.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:8","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zhao, Jinhua","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhao, Jinhua","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:9","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zhou, Yuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhou, Yuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:10","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}}],"participationCoverage":{"total":11,"returned":11,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.13015","snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z","basis":"publisher-page-metadata","sourceLinks":49,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c95b2d566ae4f0801c86219a9c2a8d0e282ed0366f6960b5c9481d4563d2ab34/source"}},"canonicalUrl":"https://escapemeridian.com/readings/from-questions-to-decisions-benchmarking-how-llm-agents-acquire-c95b2d56","formats":{"html":"/readings/from-questions-to-decisions-benchmarking-how-llm-agents-acquire-c95b2d56","markdown":"/readings/from-questions-to-decisions-benchmarking-how-llm-agents-acquire-c95b2d56.md","json":"/readings/from-questions-to-decisions-benchmarking-how-llm-agents-acquire-c95b2d56.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"c95b2d566ae4f0801c86219a9c2a8d0e282ed0366f6960b5c9481d4563d2ab34","revision":"4fb012fa-ee07-4489-8357-b1d80de3b3e3","revisionKind":"publication-id","title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","url":"https://escapemeridian.com/readings/from-questions-to-decisions-benchmarking-how-llm-agents-acquire-c95b2d56"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.13015","revision":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","revisionKind":"sha256-source-text","title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","url":"https://arxiv.org/abs/2611.13015"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"c95b2d566ae4f0801c86219a9c2a8d0e282ed0366f6960b5c9481d4563d2ab34","title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","argument":"The authors of Ask2Decide argue that LLM agents still struggle to ask people for information that changes a plan. Their benchmark tests 43 planning tasks: an operator holds facts missing from the request and records, while optimization solvers identify which facts change the optimal plan. Similar-sounding facts that do not change the plan test whether agents can distinguish useful information from irrelevant details. In 79 interaction sessions with 19 participants, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user: 43% versus 7%. The authors use those interactions to calibrate a simulated operator. Across four frontier LLMs, initial plans were infeasible on 58–98% of tasks when information was missing, versus 5–28% when all facts were supplied upfront. Agents uncovered facts that changed the optimal plan only slightly more often than facts that left it unchanged. The authors argue that this makes obtaining decision-relevant information through interaction a major bottleneck for LLM planning. The paper describes the work as ongoing and says it will be updated.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The reported evaluation covers 43 planning tasks, 79 interaction sessions with 19 participants, and four frontier LLMs; the abstract does not establish how well the results generalize beyond those settings. The authors describe the work as ongoing and say it will be updated.","speaker":{"name":"Tang, Yuhan, Fan, Ke, Zou, Xinkai, Mo, Baichuan, Cui, Kangxin, Qu, Ao, Zheng, Ou, Yu, Jiangbo, Koutsopoulos, Haris N., Zhao, Jinhua, Zhou, Yuan","roleAtSource":"Article author"},"source":{"title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.13015","publishedAt":"2026-11-09","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce Ask2Decide, a benchmark testing whether agents ask for missing information that affects planning decisions.","excerpt":"We introduce Ask2Decide, a benchmark of 43 planning tasks that evaluates whether agents ask for missing information that matters for their decisions.","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"In the reported participant sessions, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user.","excerpt":"human replies omit the correcting fact far more often than replies from an uncalibrated simulated user (43% versus 7%).","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"Across four evaluated frontier LLMs, initial plans were more often infeasible when information was missing than when all facts were provided upfront.","excerpt":"initial plans are infeasible on 58-98% of tasks when information is missing, compared with 5-28% when all facts are provided upfront.","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"The authors report that agents only slightly more often uncovered facts that changed the optimal plan than facts that did not change it.","excerpt":"Agents uncover facts that change the optimal plan only slightly more often than facts that leave it unchanged (39-52% versus 32-41%).","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"},{"text":"The authors characterize obtaining decision-relevant information through interaction as a major bottleneck for LLM planning.","excerpt":"These results show that obtaining decision-relevant information through interaction remains a major bottleneck for LLM planning.","url":"https://arxiv.org/abs/2611.13015","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.13015","checkedAt":"2026-11-12T02:24:13.234Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"As this ongoing work is updated, do agents become better at uncovering facts that change the optimal plan than facts that leave it unchanged?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","summary":"The authors of Ask2Decide argue that LLM agents still struggle to ask people for information that changes a plan. Their benchmark tests 43 planning tasks: an operator holds facts missing from the request and records, while optimization solvers identify which facts change the optimal plan. Similar-sounding facts that do not change the plan test whether agents can distinguish useful information from irrelevant details. In 79 interaction sessions with 19 participants, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user: 43% versus 7%. The authors use those interactions to calibrate a simulated operator. Across four frontier LLMs, initial plans were infeasible on 58–98% of tasks when information was missing, versus 5–28% when all facts were supplied upfront. Agents uncovered facts that changed the optimal plan only slightly more often than facts that left it unchanged. The authors argue that this makes obtaining decision-relevant information through interaction a major bottleneck for LLM planning. The paper describes the work as ongoing and says it will be updated.","question":"As this ongoing work is updated, do agents become better at uncovering facts that change the optimal plan than facts that leave it unchanged?","claims":[{"text":"The authors introduce Ask2Decide, a benchmark testing whether agents ask for missing information that affects planning decisions.","excerpt":"We introduce Ask2Decide, a benchmark of 43 planning tasks that evaluates whether agents ask for missing information that matters for their decisions.","support":"source-stated"},{"text":"In the reported participant sessions, human replies after an incorrect plan omitted the correcting fact more often than replies from an uncalibrated simulated user.","excerpt":"human replies omit the correcting fact far more often than replies from an uncalibrated simulated user (43% versus 7%).","support":"source-stated"},{"text":"Across four evaluated frontier LLMs, initial plans were more often infeasible when information was missing than when all facts were provided upfront.","excerpt":"initial plans are infeasible on 58-98% of tasks when information is missing, compared with 5-28% when all facts are provided upfront.","support":"source-stated"},{"text":"The authors report that agents only slightly more often uncovered facts that changed the optimal plan than facts that did not change it.","excerpt":"Agents uncover facts that change the optimal plan only slightly more often than facts that leave it unchanged (39-52% versus 32-41%).","support":"source-stated"},{"text":"The authors characterize obtaining decision-relevant information through interaction as a major bottleneck for LLM planning.","excerpt":"These results show that obtaining decision-relevant information through interaction remains a major bottleneck for LLM planning.","support":"source-stated"}],"limitations":["The reported evaluation covers 43 planning tasks, 79 interaction sessions with 19 participants, and four frontier LLMs; the abstract does not establish how well the results generalize beyond those settings.","The authors describe the work as ongoing and say it will be updated."],"mediaId":"590f86c72f2f70d65b2261b1b666cd78848a91f8aa6f740da5f588aa8bd76099","correctionReason":null,"id":"c95b2d566ae4f0801c86219a9c2a8d0e282ed0366f6960b5c9481d4563d2ab34","publicationId":"4fb012fa-ee07-4489-8357-b1d80de3b3e3","version":1,"sourceUrl":"https://arxiv.org/abs/2611.13015","sourceTitle":"From Questions to Decisions: Benchmarking How LLM Agents Acquire Decision-Relevant Information from Human Beings","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-09","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-11T05:06:49.684Z","createdAt":"2026-11-12T00:17:45.770Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"590f86c72f2f70d65b2261b1b666cd78848a91f8aa6f740da5f588aa8bd76099","kind":"article","url":"https://arxiv.org/abs/2611.13015","sourceUrl":"https://arxiv.org/abs/2611.13015","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T00:16:42.607Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.13015","checkedAt":"2026-11-12T02:24:13.234Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"590f86c72f2f70d65b2261b1b666cd78848a91f8aa6f740da5f588aa8bd76099","confidence":0.8500000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"b2079236ef484c7f3ca41e381e7e4196ba08d718b3db2cb50210a2e01f154746","decidedAt":"2026-11-12T00:16:43.574Z"},"provenance":{"textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observationId":"b24f2a1ea101f5ac270fc98f1550fb0f0a4313b3f94275e9543cd2ca4beefec5","retrievedAt":"2026-11-12T00:16:42.470Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"0ac6a20a-7164-4da7-8ad9-c45967018258"},"reviewer":{"model":"gpt-6-luna","runId":"29637b41-0d91-4826-81ea-f0eae7c7ac0e"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-12T00:17:45.770Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Tang, Yuhan","url":null,"kind":"unknown"},{"name":"Fan, Ke","url":null,"kind":"unknown"},{"name":"Zou, Xinkai","url":null,"kind":"unknown"},{"name":"Mo, Baichuan","url":null,"kind":"unknown"},{"name":"Cui, Kangxin","url":null,"kind":"unknown"},{"name":"Qu, Ao","url":null,"kind":"unknown"},{"name":"Zheng, Ou","url":null,"kind":"unknown"},{"name":"Yu, Jiangbo","url":null,"kind":"unknown"},{"name":"Koutsopoulos, Haris N.","url":null,"kind":"unknown"},{"name":"Zhao, Jinhua","url":null,"kind":"unknown"},{"name":"Zhou, Yuan","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Tang, Yuhan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Tang, Yuhan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Fan, Ke","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Fan, Ke","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zou, Xinkai","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zou, Xinkai","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Mo, Baichuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Mo, Baichuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Cui, Kangxin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cui, Kangxin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Qu, Ao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qu, Ao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zheng, Ou","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zheng, Ou","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Yu, Jiangbo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yu, Jiangbo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Koutsopoulos, Haris N.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Koutsopoulos, Haris N.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:8","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zhao, Jinhua","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhao, Jinhua","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:9","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}},{"actor":{"name":"Zhou, Yuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhou, Yuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.13015","locator":"head/meta[name=author|citation_author]"},"assertionId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c:participation:10","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-12T02:24:13.234Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","sourceUrl":"https://arxiv.org/abs/2611.13015","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z"}}],"participationCoverage":{"total":11,"returned":11,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.13015","snapshotId":"a830e6f5ccbd9a74e762717d59022ab75ddb285cc08a707e81564faa9bc3fa2c","contentSha256":"e8963d95b5c6ac424f32852af1ca237b00cd49b188b9b47285160ca4439f3580","textHash":"e9c702526714e7e161ed4ee593ca6106e0f3b14abca4797c5ad31fe2f8646dc3","observedAt":"2026-11-12T02:24:13.234Z","basis":"publisher-page-metadata","sourceLinks":49,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/c95b2d566ae4f0801c86219a9c2a8d0e282ed0366f6960b5c9481d4563d2ab34/source"}}}],"audits":[{"publicationId":"4fb012fa-ee07-4489-8357-b1d80de3b3e3","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The findings and the bottleneck conclusion are consistently attributed to the paper's authors and presented as their reported results, not as established fact. The arXiv author list matches the source.","excerpt":"Authors:Yuhan Tang, Ke Fan, Xinkai Zou, Baichuan Mo, Kangxin Cui, Ao Qu, Ou Zheng, Jiangbo Yu, Haris N. Koutsopoulos, Jinhua Zhao, Yuan Zhou"},{"dimension":"summary","status":"uncertain","explanation":"The summary is accurate but omits context a lay reader needs. It does not say what these agents do: the abstract's examples are scheduling drivers and assigning nurses, where requests are incomplete or outdated. It also drops the authors' reason the benchmark exists: earlier tests used fully specified problems or uncalibrated simulated users. Without that, the 43% versus 7% result reads as a curiosity, not evidence that simulated users are too forthcoming. 'Far more often' is weakened to 'more often', and 'infeasible' and 'operator' are unexplained. Smallest repair: add one sentence with the use-case examples and the gap in existing benchmarks.","excerpt":"Existing benchmarks typically provide fully specified problems or rely on uncalibrated LLM-simulated users, leaving this ability insufficiently tested."},{"dimension":"date","status":"pass","explanation":"The originalPublished value of 2026-11-09 matches the v1 submission date on arXiv. The later feed date is not the original publication date.","excerpt":"[Submitted on 9 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Every claim and number matches the abstract verbatim: 43 tasks, 79 sessions, 19 participants, 43% versus 7%, 58-98% versus 5-28%, and slightly more often. The limitations and the question follow from the source without adding benefits or costs. The question is grounded in the authors' note that the work is ongoing and will be updated.","excerpt":"Agents uncover facts that change the optimal plan only slightly more often than facts that leave it unchanged (39-52% versus 32-41%)."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the arXiv abstract page. It is the original source and the most relevant choice.","excerpt":null}],"createdAt":"2026-11-12T00:17:45.770Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}