# AOS-Bench: Can LLM Agents Manage Dynamic Workloads? [Canonical reading](https://escapemeridian.com/readings/aos-bench-can-llm-agents-manage-dynamic-workloads-a7013a77) Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su and Hongning Wang introduce AOS-Bench to test coding agents that manage a stream of tasks sharing a machine, rather than handling one task at a time. The benchmark measures whether verified tasks arriving over time are delivered under shared CPU and memory limits, and compares this with the same tasks run one at a time. Across five model–harness pairs, the authors report that performance on isolated tasks does not reliably carry over to streams: every system loses 13 to 23 points on heavy workloads. They trace much of the gap to task intake; serial dispatch leaves 73% of heavy-workload tasks unclaimed. Yet elaborate management is not automatically better: fixed concurrency of two comes within four points of the best agent-managed policy on medium and heavy workloads. The authors argue that workload management is a distinct capability worth measuring. Their results make task intake and shared-resource management part of the evaluation, while leaving open how well these findings extend beyond the systems and workloads tested. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Oct 23, 2026 (2026-10-23). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T20:46:28.439Z. First observed: 2026-10-28T04:04:32.994Z. Retrieved: 2026-10-30T19:35:30.346Z. - [AOS-Bench: Can LLM Agents Manage Dynamic Workloads?](https://arxiv.org/abs/2610.33178) ## Limits The abstract reports results across five model–harness pairs; it does not establish how well the findings generalize to other systems or workloads. The evidence provided is the paper’s abstract, which does not give details of the workload construction or evaluation procedure. ## Open question How well would AOS-Bench’s findings about task intake and management extend to other models, harnesses and workloads? ## Source-stated The authors introduce AOS-Bench to evaluate agents managing tasks that arrive over time under shared CPU and memory limits, with one-at-a-time runs as a reference. > verified tasks arrive over time and must be delivered under shared CPU and memory limits, and the same tasks run one at a time serve as a per-system reference. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated Across five model–harness pairs, isolated task performance does not carry over evenly to workloads with multiple pending tasks. > Across five model--harness pairs, isolated task success does not carry over evenly. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated The authors report that serial dispatch leaves many heavy-workload tasks unclaimed, and attribute much of the performance gap to task intake. > Execution-policy interventions trace much of this gap to task intake: serial dispatch leaves 73\\% of heavy-workload tasks unclaimed. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated A fixed concurrency of two performed close to the best agent-managed policy on medium and heavy workloads in the reported tests. > A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated The authors present workload management as a distinct capability for agents and AOS-Bench as a way to measure it. > These results identify workload management as a distinct capability for agents, and AOS-Bench as a way to measure it. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source and contributor context Zhu, Erle, Zhang, Genrui, Hu, Keyang, Su, Mingyu, Wang, Hongning · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## CPU The central processing unit: a general-purpose processor that carries out a computer's instructions. - [Definition reference](https://aws.amazon.com/what-is/cpu/) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: e92fcf96-4556-473a-837f-fb981b0a5b31. Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T20:46:28.439Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 792fd1742f503c63346ba3b4937be133f0a6409ad6991c4682753b3a19e6a9ac. Source text hash: cd3057ae44119eb612b7412819c7a94e73c66aca5a1b05d3427c5bc1c91549a8. ## Revision and source-check history ### Version 1 — 2026-11-10T20:46:28.439Z Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su and Hongning Wang introduce AOS-Bench to test coding agents that manage a stream of tasks sharing a machine, rather than handling one task at a time. The benchmark measures whether verified tasks arriving over time are delivered under shared CPU and memory limits, and compares this with the same tasks run one at a time. Across five model–harness pairs, the authors report that performance on isolated tasks does not reliably carry over to streams: every system loses 13 to 23 points on heavy workloads. They trace much of the gap to task intake; serial dispatch leaves 73% of heavy-workload tasks unclaimed. Yet elaborate management is not automatically better: fixed concurrency of two comes within four points of the best agent-managed policy on medium and heavy workloads. The authors argue that workload management is a distinct capability worth measuring. Their results make task intake and shared-resource management part of the evaluation, while leaving open how well these findings extend beyond the systems and workloads tested. ## Original source arXiv cs.AI · Article Original source date: Oct 23, 2026 (2026-10-23). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T20:46:28.439Z. First observed: 2026-10-28T04:04:32.994Z. Retrieved: 2026-10-30T19:35:30.346Z. - [AOS-Bench: Can LLM Agents Manage Dynamic Workloads?](https://arxiv.org/abs/2610.33178) ## Limits The abstract reports results across five model–harness pairs; it does not establish how well the findings generalize to other systems or workloads. The evidence provided is the paper’s abstract, which does not give details of the workload construction or evaluation procedure. ## Open question How well would AOS-Bench’s findings about task intake and management extend to other models, harnesses and workloads? ## Source-stated The authors introduce AOS-Bench to evaluate agents managing tasks that arrive over time under shared CPU and memory limits, with one-at-a-time runs as a reference. > verified tasks arrive over time and must be delivered under shared CPU and memory limits, and the same tasks run one at a time serve as a per-system reference. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated Across five model–harness pairs, isolated task performance does not carry over evenly to workloads with multiple pending tasks. > Across five model--harness pairs, isolated task success does not carry over evenly. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated The authors report that serial dispatch leaves many heavy-workload tasks unclaimed, and attribute much of the performance gap to task intake. > Execution-policy interventions trace much of this gap to task intake: serial dispatch leaves 73\\% of heavy-workload tasks unclaimed. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated A fixed concurrency of two performed close to the best agent-managed policy on medium and heavy workloads in the reported tests. > A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source-stated The authors present workload management as a distinct capability for agents and AOS-Bench as a way to measure it. > These results identify workload management as a distinct capability for agents, and AOS-Bench as a way to measure it. - [Supporting source](https://arxiv.org/abs/2610.33178) ## Source and contributor context Zhu, Erle, Zhang, Genrui, Hu, Keyang, Su, Mingyu, Wang, Hongning · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## CPU The central processing unit: a general-purpose processor that carries out a computer's instructions. - [Definition reference](https://aws.amazon.com/what-is/cpu/) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## TIME TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services. - [Explore this context](/entities/time) - [Reference](https://time.com/) ## Editorial provenance Version 1. Publication ID: e92fcf96-4556-473a-837f-fb981b0a5b31. Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T20:46:28.439Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 792fd1742f503c63346ba3b4937be133f0a6409ad6991c4682753b3a19e6a9ac. Source text hash: cd3057ae44119eb612b7412819c7a94e73c66aca5a1b05d3427c5bc1c91549a8. ### uncertain — 2026-11-10T20:46:28.439Z — anthropic/claude-opus-5.5 Publication: e92fcf96-4556-473a-837f-fb981b0a5b31 attribution — pass: The reading correctly names the five listed authors and attributes the benchmark, findings and argument to them as reported claims, not established fact. > Authors:Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su, Hongning Wang summary — uncertain: Central terms go unexplained for a general reader: 'model–harness pairs', 'serial dispatch' (claiming one task at a time) and 'fixed concurrency of two' (always running two tasks at once). Smallest repair: add plain-language glosses or term-library tooltips. The summary also omits a striking finding: under every policy, median CPU use stayed below 30% while tasks waited. 'Does not reliably carry over' slightly shifts the source's 'unevenly' point, which is better illustrated by its medium-workload example (54% vs 68% for systems 0.6 points apart). > under every policy median CPU use stays below 30\\% of the provisioned cores while tasks wait date — pass: originalPublished 2026-10-23 matches the v1 submission date. It is not the later v2 revision or the feed date. > \[Submitted on 23 Oct 2026 (v1), last revised 27 Oct 2026 (this version, v2)\] support — pass: Each claim matches the quoted abstract text. The generalization question and limitations are framed as unknowns, not as invented facts. No unsupported benefits or costs are implied. > A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct way to access the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/a7013a777ce47ca9d2ae834f0b3ea5975f152383e5365a2728b751cff459f090/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/aos-bench-can-llm-agents-manage-dynamic-workloads-a7013a77.json)