Zhu, Erle, Zhang, Genrui, Hu, Keyang, Su, Mingyu, Wang, Hongning · Oct 23, 2026
AOS-Bench: Can LLM Agents Manage Dynamic Workloads?
Markdown · JSON · Original source
Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su and Hongning Wang introduce AOS-Bench to test coding agents that manage a stream of tasks sharing a machine, rather than handling one task at a . The measures whether verified tasks arriving over time are delivered under shared and memory limits, and compares this with the same tasks run one at a time. Across five model–harness pairs, the authors report that performance on isolated tasks does not reliably carry over to streams: every system loses 13 to 23 points on heavy workloads. They trace much of the gap to task intake; serial dispatch leaves 73% of heavy-workload tasks unclaimed. Yet elaborate management is not automatically better: fixed concurrency of two comes within four points of the best agent-managed policy on medium and heavy workloads. The authors argue that workload management is a distinct capability worth measuring. Their results make task intake and shared-resource management part of the evaluation, while leaving open how well these findings extend beyond the systems and workloads tested.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 23, 2026 (2026-10-23).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T20:46:28.439Z. First observed: 2026-10-28T04:04:32.994Z. Retrieved: 2026-10-30T19:35:30.346Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract reports results across five model–harness pairs; it does not establish how well the findings generalize to other systems or workloads. The evidence provided is the paper’s abstract, which does not give details of the workload construction or evaluation procedure.
Open question
How well would AOS-Bench’s findings about task intake and management extend to other models, harnesses and workloads?
Source-stated
The authors introduce AOS-Bench to evaluate agents managing tasks that arrive over time under shared CPU and memory limits, with one-at-a-time runs as a reference.
verified tasks arrive over time and must be delivered under shared CPU and memory limits, and the same tasks run one at a time serve as a per-system reference.
Source-stated
Across five model–harness pairs, isolated task performance does not carry over evenly to workloads with multiple pending tasks.
Across five model--harness pairs, isolated task success does not carry over evenly.
Source-stated
The authors report that serial dispatch leaves many heavy-workload tasks unclaimed, and attribute much of the performance gap to task intake.
Execution-policy interventions trace much of this gap to task intake: serial dispatch leaves 73\% of heavy-workload tasks unclaimed.
Source-stated
A fixed concurrency of two performed close to the best agent-managed policy on medium and heavy workloads in the reported tests.
A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads
Source-stated
The authors present workload management as a distinct capability for agents and AOS-Bench as a way to measure it.
These results identify workload management as a distinct capability for agents, and AOS-Bench as a way to measure it.
Source and contributor context
Zhu, Erle, Zhang, Genrui, Hu, Keyang, Su, Mingyu, Wang, Hongning · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
CPU
The central processing unit: a general-purpose processor that carries out a computer's instructions.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: e92fcf96-4556-473a-837f-fb981b0a5b31.
Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T20:46:28.439Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 792fd1742f503c63346ba3b4937be133f0a6409ad6991c4682753b3a19e6a9ac. Source text hash: cd3057ae44119eb612b7412819c7a94e73c66aca5a1b05d3427c5bc1c91549a8.
Revision and source-check history
Version 1 · 2026-11-10T20:46:28.439Z
AOS-Bench: Can LLM Agents Manage Dynamic Workloads?
Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su and Hongning Wang introduce AOS-Bench to test coding agents that manage a stream of tasks sharing a machine, rather than handling one task at a . The measures whether verified tasks arriving over time are delivered under shared and memory limits, and compares this with the same tasks run one at a time. Across five model–harness pairs, the authors report that performance on isolated tasks does not reliably carry over to streams: every system loses 13 to 23 points on heavy workloads. They trace much of the gap to task intake; serial dispatch leaves 73% of heavy-workload tasks unclaimed. Yet elaborate management is not automatically better: fixed concurrency of two comes within four points of the best agent-managed policy on medium and heavy workloads. The authors argue that workload management is a distinct capability worth measuring. Their results make task intake and shared-resource management part of the evaluation, while leaving open how well these findings extend beyond the systems and workloads tested.
Original source
cs.AI · Article
Source details
Original source date: Oct 23, 2026 (2026-10-23).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T20:46:28.439Z. First observed: 2026-10-28T04:04:32.994Z. Retrieved: 2026-10-30T19:35:30.346Z.
Limits
The abstract reports results across five model–harness pairs; it does not establish how well the findings generalize to other systems or workloads. The evidence provided is the paper’s abstract, which does not give details of the workload construction or evaluation procedure.
Open question
How well would AOS-Bench’s findings about task intake and management extend to other models, harnesses and workloads?
Source-stated
The authors introduce AOS-Bench to evaluate agents managing tasks that arrive over time under shared CPU and memory limits, with one-at-a-time runs as a reference.
verified tasks arrive over time and must be delivered under shared CPU and memory limits, and the same tasks run one at a time serve as a per-system reference.
Source-stated
Across five model–harness pairs, isolated task performance does not carry over evenly to workloads with multiple pending tasks.
Across five model--harness pairs, isolated task success does not carry over evenly.
Source-stated
The authors report that serial dispatch leaves many heavy-workload tasks unclaimed, and attribute much of the performance gap to task intake.
Execution-policy interventions trace much of this gap to task intake: serial dispatch leaves 73\% of heavy-workload tasks unclaimed.
Source-stated
A fixed concurrency of two performed close to the best agent-managed policy on medium and heavy workloads in the reported tests.
A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads
Source-stated
The authors present workload management as a distinct capability for agents and AOS-Bench as a way to measure it.
These results identify workload management as a distinct capability for agents, and AOS-Bench as a way to measure it.
Source and contributor context
Zhu, Erle, Zhang, Genrui, Hu, Keyang, Su, Mingyu, Wang, Hongning · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
CPU
The central processing unit: a general-purpose processor that carries out a computer's instructions.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: e92fcf96-4556-473a-837f-fb981b0a5b31.
Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T20:46:28.439Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 792fd1742f503c63346ba3b4937be133f0a6409ad6991c4682753b3a19e6a9ac. Source text hash: cd3057ae44119eb612b7412819c7a94e73c66aca5a1b05d3427c5bc1c91549a8.
uncertain · 2026-11-10T20:46:28.439Z · anthropic/claude-opus-5.5
Publication: e92fcf96-4556-473a-837f-fb981b0a5b31
attribution — pass: The reading correctly names the five listed authors and attributes the benchmark, findings and argument to them as reported claims, not established fact.
Authors:Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su, Hongning Wang
summary — uncertain: Central terms go unexplained for a general reader: 'model–harness pairs', 'serial dispatch' (claiming one task at a time) and 'fixed concurrency of two' (always running two tasks at once). Smallest repair: add plain-language glosses or term-library tooltips. The summary also omits a striking finding: under every policy, median CPU use stayed below 30% while tasks waited. 'Does not reliably carry over' slightly shifts the source's 'unevenly' point, which is better illustrated by its medium-workload example (54% vs 68% for systems 0.6 points apart).
under every policy median CPU use stays below 30\% of the provisioned cores while tasks wait
date — pass: originalPublished 2026-10-23 matches the v1 submission date. It is not the later v2 revision or the feed date.
[Submitted on 23 Oct 2026 (v1), last revised 27 Oct 2026 (this version, v2)]
support — pass: Each claim matches the quoted abstract text. The generalization question and limitations are framed as unknowns, not as invented facts. No unsupported benefits or costs are implied.
A fixed concurrency of two comes within four points of the best agent-managed policy on both medium and heavy workloads
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct way to access the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceThe central processing unit: a general-purpose processor that carries out a computer's instructions.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceThe central processing unit: a general-purpose processor that carries out a computer's instructions.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv