· Dec 31, 2024
Google DeepMind reports lower success on composed web tasks
Markdown · JSON · Original source
’s abstract, published December 31, 2024, reports that language model agents perform much worse on web tasks combining several tasks than on individual base tasks. The authors introduce CompWoB, a of 50 compositional web automation tasks, to study performance under more realistic combinations. Prompted agents achieved a 94.0% average success rate on base tasks but 24.9% on compositional tasks. Agents transferred after training only on base tasks fell from 85.4% to 54.8%, a smaller drop. The authors also report that HTML-T5++ reached 95.2% on MiniWoB and the best performance on CompWoB, at 61.5%. But performance declined further when instruction compositions changed the order of combinations. The results make the authors’ point more specific than a general claim about agent capability: success on separate tasks may not carry over when real-world work requires those tasks in sequence, and changing their order poses a further challenge. The authors conclude that real-world deployment requires agents robust and generalizable to task compositionality. The abstract does not describe the tasks or evaluation procedure in detail.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
· Article
Source details
Original source date: Dec 31, 2024 (2024-12-31T00:00:00+00:00).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-16T04:28:25.203Z. First observed: 2026-11-02T17:07:47.604Z. Retrieved: 2026-11-02T17:13:26.815Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract does not describe the tasks or evaluation procedure in detail. Text as retrieved on November 2, 2026; originally published December 31, 2024.
Open question
Which changes in task order or instruction composition account for the further performance decline the authors report?
Source-stated
The authors introduce CompWoB, a benchmark of 50 compositional web automation tasks.
we introduce a new benchmark, called CompWoB -- 50 new compositional web automation tasks reflecting more realistic assumptions.
Source-stated
Prompted language model agents achieved a 94.0% average success rate on base tasks and 24.9% on compositional tasks.
existing prompted LMAs (gpt-3.5-turbo or gpt-4) achieve 94.0% average success rate on base tasks, their performance degrades to 24.9% success rate on compositional tasks.
Source-stated
Transferred agents, finetuned only on base tasks, dropped from 85.4% to 54.8%.
transferred LMAs (finetuned only on base tasks) show less generalization gap, dropping from 85.4% to 54.8%.
Source-stated
The authors report that HTML-T5++ surpassed human-level performance on MiniWoB and achieved the best zero-shot performance on CompWoB.
HTML-T5++, that surpasses human-level performance (95.2%) on MiniWoB, and achieves the best zero-shot performance on CompWoB (61.5%).
Source-stated
The authors report further performance declines when instruction compositions change the order of combinations.
their performance further degrades under different instruction compositions changing combinational order.
Source-stated
The authors argue that robust and generalizable language model agents are necessary for real-world deployment involving task compositionality.
emphasize the necessity of building LMAs that are robust and generalizable to task compositionality for real-world deployment.
Source and contributor context
Google DeepMind · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Zero-shot
Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).
Google DeepMind
Google DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.
Editorial provenance
Version 1. Publication ID: 181d22f2-074e-3873-a8b5-1643f5bb8180.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T04:23:47.036Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 098429c150b6c71dbc3c31bb5bd497fe4e54b55a0a2ee5c7ae4a593bb75a45e9. Source text hash: 0e68ee2d23b865900a1f97ffb9b92ed780786c2c86f2847ee61ae42010cd035b.
Revision and source-check history
Version 1 · 2026-11-16T04:28:25.203Z
Google DeepMind reports lower success on composed web tasks
’s abstract, published December 31, 2024, reports that language model agents perform much worse on web tasks combining several tasks than on individual base tasks. The authors introduce CompWoB, a of 50 compositional web automation tasks, to study performance under more realistic combinations. Prompted agents achieved a 94.0% average success rate on base tasks but 24.9% on compositional tasks. Agents transferred after training only on base tasks fell from 85.4% to 54.8%, a smaller drop. The authors also report that HTML-T5++ reached 95.2% on MiniWoB and the best performance on CompWoB, at 61.5%. But performance declined further when instruction compositions changed the order of combinations. The results make the authors’ point more specific than a general claim about agent capability: success on separate tasks may not carry over when real-world work requires those tasks in sequence, and changing their order poses a further challenge. The authors conclude that real-world deployment requires agents robust and generalizable to task compositionality. The abstract does not describe the tasks or evaluation procedure in detail.
Original source
· Article
Source details
Original source date: Dec 31, 2024 (2024-12-31T00:00:00+00:00).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-16T04:28:25.203Z. First observed: 2026-11-02T17:07:47.604Z. Retrieved: 2026-11-02T17:13:26.815Z.
Limits
The abstract does not describe the tasks or evaluation procedure in detail. Text as retrieved on November 2, 2026; originally published December 31, 2024.
Open question
Which changes in task order or instruction composition account for the further performance decline the authors report?
Source-stated
The authors introduce CompWoB, a benchmark of 50 compositional web automation tasks.
we introduce a new benchmark, called CompWoB -- 50 new compositional web automation tasks reflecting more realistic assumptions.
Source-stated
Prompted language model agents achieved a 94.0% average success rate on base tasks and 24.9% on compositional tasks.
existing prompted LMAs (gpt-3.5-turbo or gpt-4) achieve 94.0% average success rate on base tasks, their performance degrades to 24.9% success rate on compositional tasks.
Source-stated
Transferred agents, finetuned only on base tasks, dropped from 85.4% to 54.8%.
transferred LMAs (finetuned only on base tasks) show less generalization gap, dropping from 85.4% to 54.8%.
Source-stated
The authors report that HTML-T5++ surpassed human-level performance on MiniWoB and achieved the best zero-shot performance on CompWoB.
HTML-T5++, that surpasses human-level performance (95.2%) on MiniWoB, and achieves the best zero-shot performance on CompWoB (61.5%).
Source-stated
The authors report further performance declines when instruction compositions change the order of combinations.
their performance further degrades under different instruction compositions changing combinational order.
Source-stated
The authors argue that robust and generalizable language model agents are necessary for real-world deployment involving task compositionality.
emphasize the necessity of building LMAs that are robust and generalizable to task compositionality for real-world deployment.
Source and contributor context
Google DeepMind · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Zero-shot
Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).
Google DeepMind
Google DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.
Editorial provenance
Version 1. Publication ID: 181d22f2-074e-3873-a8b5-1643f5bb8180.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T04:23:47.036Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 098429c150b6c71dbc3c31bb5bd497fe4e54b55a0a2ee5c7ae4a593bb75a45e9. Source text hash: 0e68ee2d23b865900a1f97ffb9b92ed780786c2c86f2847ee61ae42010cd035b.
uncertain · 2026-11-16T04:23:47.036Z · anthropic/claude-opus-5.5
Publication: 181d22f2-074e-3873-a8b5-1643f5bb8180
attribution — pass: The reading attributes the findings to the abstract on Google DeepMind's publication page and to 'the authors'. The page lists four authors and the TMLR venue. Naming the authors is not required. All figures and conclusions stay attributed to the source.
Hiroki Furuta, Yutaka Matsuo, Aleksandra Faust, Izzeddin Gur
summary — uncertain: The summary says HTML-T5++ 'reached 95.2% on MiniWoB' but never explains what MiniWoB is. A general reader cannot tell that it is the established set of individual web tasks underlying the base-task results. The summary also drops the source's point that 95.2% surpasses human-level performance, which appears only in the claims. Smallest repair: add one clause explaining MiniWoB, and restore that the abstract calls this result above human level. Benchmark and zero-shot are defined beside the reading. The title and question are fair.
HTML-T5++, that surpasses human-level performance (95.2%) on MiniWoB
date — pass: The page dateline reads December 31, 2024, which matches originalPublished. The reading states that date at day precision and adds no hindsight. The page was modified in December 2025, but the cited abstract is plainly the original publication content. The evidence kind is publisher-page, so no archive disclosure is needed.
December 31, 2024
support — pass: All the figures (94.0% to 24.9%, 85.4% to 54.8%, 95.2%, 61.5%) and the order-sensitivity finding match the abstract verbatim. The hedged inference that success on separate tasks may not carry over to sequenced real-world work follows from the abstract's framing of real-world applications as combinations of tasks. The later page edit does not affect the abstract text cited.
their performance on real-world applications that often involve combinations of tasks is still underexplored
media — pass: No media was selected, and no verified media candidates were supplied, so there is no identity or relevance problem.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Google DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.
Explore Google DeepMindGoogle DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.
Explore Google DeepMindA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceDoing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).
Definition referenceGoogle DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.
Explore Google DeepMindA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceGoogle DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.
Explore Google DeepMindA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceDoing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).
Definition referenceGoogle DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.
Explore Google DeepMindA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition reference