# Google DeepMind reports lower success on composed web tasks

[Canonical reading](https://escapemeridian.com/readings/google-deepmind-reports-lower-success-on-composed-web-tasks-707ce0ce)

Google DeepMind’s abstract, published December 31, 2024, reports that language model agents perform much worse on web tasks combining several tasks than on individual base tasks. The authors introduce CompWoB, a benchmark of 50 compositional web automation tasks, to study performance under more realistic combinations. Prompted agents achieved a 94.0% average success rate on base tasks but 24.9% on compositional tasks. Agents transferred after training only on base tasks fell from 85.4% to 54.8%, a smaller drop. The authors also report that HTML-T5++ reached 95.2% on MiniWoB and the best zero-shot performance on CompWoB, at 61.5%. But performance declined further when instruction compositions changed the order of combinations. The results make the authors’ point more specific than a general claim about agent capability: success on separate tasks may not carry over when real-world work requires those tasks in sequence, and changing their order poses a further challenge. The authors conclude that real-world deployment requires agents robust and generalizable to task compositionality. The abstract does not describe the tasks or evaluation procedure in detail.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

Google DeepMind · Article

Original source date: Dec 31, 2024 (2024-12-31T00:00:00+00:00).

Date basis: Date stated on the publisher’s page.

Added to Escape Meridian: 2026-11-16T04:28:25.203Z. First observed: 2026-11-02T17:07:47.604Z. Retrieved: 2026-11-02T17:13:26.815Z.

- [Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web](https://deepmind.google/research/publications/46840/)

## Limits

The abstract does not describe the benchmark tasks or evaluation procedure in detail. Text as retrieved on November 2, 2026; originally published December 31, 2024.


## Open question

Which changes in task order or instruction composition account for the further performance decline the authors report?


## Source-stated

The authors introduce CompWoB, a benchmark of 50 compositional web automation tasks.

> we introduce a new benchmark, called CompWoB -- 50 new compositional web automation tasks reflecting more realistic assumptions.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

Prompted language model agents achieved a 94.0% average success rate on base tasks and 24.9% on compositional tasks.

> existing prompted LMAs (gpt-3.5-turbo or gpt-4) achieve 94.0% average success rate on base tasks, their performance degrades to 24.9% success rate on compositional tasks.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

Transferred agents, finetuned only on base tasks, dropped from 85.4% to 54.8%.

> transferred LMAs (finetuned only on base tasks) show less generalization gap, dropping from 85.4% to 54.8%.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

The authors report that HTML-T5++ surpassed human-level performance on MiniWoB and achieved the best zero-shot performance on CompWoB.

> HTML-T5++, that surpasses human-level performance (95.2%) on MiniWoB, and achieves the best zero-shot performance on CompWoB (61.5%).

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

The authors report further performance declines when instruction compositions change the order of combinations.

> their performance further degrades under different instruction compositions changing combinational order.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

The authors argue that robust and generalizable language model agents are necessary for real-world deployment involving task compositionality.

> emphasize the necessity of building LMAs that are robust and generalizable to task compositionality for real-world deployment.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source and contributor context

Google DeepMind · Publisher

The publisher is identified here. No individual author has been inferred.


## Source media

Media retrieval status: unchecked.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## Zero-shot

Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).

- [Definition reference](https://developers.google.com/machine-learning/glossary#zero-shot-prompting)

## Google DeepMind

Google DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.

- [Explore this context](/entities/google-deepmind)
- [Reference](https://deepmind.google/blog/)
- [Reference](https://deepmind.google/science/alphafold/)

## Editorial provenance

Version 1. Publication ID: 181d22f2-074e-3873-a8b5-1643f5bb8180.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T04:23:47.036Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical.

Observation: 098429c150b6c71dbc3c31bb5bd497fe4e54b55a0a2ee5c7ae4a593bb75a45e9. Source text hash: 0e68ee2d23b865900a1f97ffb9b92ed780786c2c86f2847ee61ae42010cd035b.


## Revision and source-check history

### Version 1 — 2026-11-16T04:28:25.203Z

Google DeepMind’s abstract, published December 31, 2024, reports that language model agents perform much worse on web tasks combining several tasks than on individual base tasks. The authors introduce CompWoB, a benchmark of 50 compositional web automation tasks, to study performance under more realistic combinations. Prompted agents achieved a 94.0% average success rate on base tasks but 24.9% on compositional tasks. Agents transferred after training only on base tasks fell from 85.4% to 54.8%, a smaller drop. The authors also report that HTML-T5++ reached 95.2% on MiniWoB and the best zero-shot performance on CompWoB, at 61.5%. But performance declined further when instruction compositions changed the order of combinations. The results make the authors’ point more specific than a general claim about agent capability: success on separate tasks may not carry over when real-world work requires those tasks in sequence, and changing their order poses a further challenge. The authors conclude that real-world deployment requires agents robust and generalizable to task compositionality. The abstract does not describe the tasks or evaluation procedure in detail.

## Original source

Google DeepMind · Article

Original source date: Dec 31, 2024 (2024-12-31T00:00:00+00:00).

Date basis: Date stated on the publisher’s page.

Added to Escape Meridian: 2026-11-16T04:28:25.203Z. First observed: 2026-11-02T17:07:47.604Z. Retrieved: 2026-11-02T17:13:26.815Z.

- [Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web](https://deepmind.google/research/publications/46840/)

## Limits

The abstract does not describe the benchmark tasks or evaluation procedure in detail. Text as retrieved on November 2, 2026; originally published December 31, 2024.


## Open question

Which changes in task order or instruction composition account for the further performance decline the authors report?


## Source-stated

The authors introduce CompWoB, a benchmark of 50 compositional web automation tasks.

> we introduce a new benchmark, called CompWoB -- 50 new compositional web automation tasks reflecting more realistic assumptions.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

Prompted language model agents achieved a 94.0% average success rate on base tasks and 24.9% on compositional tasks.

> existing prompted LMAs (gpt-3.5-turbo or gpt-4) achieve 94.0% average success rate on base tasks, their performance degrades to 24.9% success rate on compositional tasks.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

Transferred agents, finetuned only on base tasks, dropped from 85.4% to 54.8%.

> transferred LMAs (finetuned only on base tasks) show less generalization gap, dropping from 85.4% to 54.8%.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

The authors report that HTML-T5++ surpassed human-level performance on MiniWoB and achieved the best zero-shot performance on CompWoB.

> HTML-T5++, that surpasses human-level performance (95.2%) on MiniWoB, and achieves the best zero-shot performance on CompWoB (61.5%).

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

The authors report further performance declines when instruction compositions change the order of combinations.

> their performance further degrades under different instruction compositions changing combinational order.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source-stated

The authors argue that robust and generalizable language model agents are necessary for real-world deployment involving task compositionality.

> emphasize the necessity of building LMAs that are robust and generalizable to task compositionality for real-world deployment.

- [Supporting source](https://deepmind.google/research/publications/46840/)

## Source and contributor context

Google DeepMind · Publisher

The publisher is identified here. No individual author has been inferred.


## Source media

Media retrieval status: unchecked.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

- [Definition reference](https://arxiv.org/abs/2211.09110)

## Zero-shot

Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).

- [Definition reference](https://developers.google.com/machine-learning/glossary#zero-shot-prompting)

## Google DeepMind

Google DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.

- [Explore this context](/entities/google-deepmind)
- [Reference](https://deepmind.google/blog/)
- [Reference](https://deepmind.google/science/alphafold/)

## Editorial provenance

Version 1. Publication ID: 181d22f2-074e-3873-a8b5-1643f5bb8180.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T04:23:47.036Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge\_admitted\_by\_model, second\_opinion\_categorical.

Observation: 098429c150b6c71dbc3c31bb5bd497fe4e54b55a0a2ee5c7ae4a593bb75a45e9. Source text hash: 0e68ee2d23b865900a1f97ffb9b92ed780786c2c86f2847ee61ae42010cd035b.


### uncertain — 2026-11-16T04:23:47.036Z — anthropic/claude-opus-5.5

Publication: 181d22f2-074e-3873-a8b5-1643f5bb8180

attribution — pass: The reading attributes the findings to the abstract on Google DeepMind's publication page and to 'the authors'. The page lists four authors and the TMLR venue. Naming the authors is not required. All figures and conclusions stay attributed to the source.

> Hiroki Furuta, Yutaka Matsuo, Aleksandra Faust, Izzeddin Gur

summary — uncertain: The summary says HTML-T5++ 'reached 95.2% on MiniWoB' but never explains what MiniWoB is. A general reader cannot tell that it is the established set of individual web tasks underlying the base-task results. The summary also drops the source's point that 95.2% surpasses human-level performance, which appears only in the claims. Smallest repair: add one clause explaining MiniWoB, and restore that the abstract calls this result above human level. Benchmark and zero-shot are defined beside the reading. The title and question are fair.

> HTML-T5++, that surpasses human-level performance (95.2%) on MiniWoB

date — pass: The page dateline reads December 31, 2024, which matches originalPublished. The reading states that date at day precision and adds no hindsight. The page was modified in December 2025, but the cited abstract is plainly the original publication content. The evidence kind is publisher-page, so no archive disclosure is needed.

> December 31, 2024

support — pass: All the figures (94.0% to 24.9%, 85.4% to 54.8%, 95.2%, 61.5%) and the order-sensitivity finding match the abstract verbatim. The hedged inference that success on separate tasks may not carry over to sequenced real-world work follows from the abstract's framing of real-world applications as combinations of tasks. The later page edit does not affect the abstract text cited.

> their performance on real-world applications that often involve combinations of tasks is still underexplored

media — pass: No media was selected, and no verified media candidates were supplied, so there is no identity or relevance problem.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/707ce0ceede7f2334e8ca25dc61e25f32f05648f090472a4e5d355616c00b9ec/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/google-deepmind-reports-lower-success-on-composed-web-tasks-707ce0ce.json)
