Lenton, Daniel, Arif, Yusha, Mahmood, Haris · Nov 10, 2026
Continual-ARC: Measuring Skill Acquisition over a Working Lifetime
Markdown · JSON · Original source
Authors Daniel Lenton, Yusha Arif and Haris Mahmood propose Continual-ARC, a for studying : whether a system can acquire skills as tasks recur over . Its stream combines tasks from three ARC-AGI editions. Tasks return after gaps with fresh instances, while new tasks arrive; learners can pay for demonstrations, submit answers or act inside an , with paid retries for wrong answers. The benchmark scores how much feedback budget each instance uses, then tests learners on familiar tasks and unseen ones. The authors say this lets memory-based and weight-based systems be compared on the same scale. In their experiments, persistence in an agent harness generally did not improve memorization without explicit expert guidance. Most harness-based methods learned less than a baseline that simply cached task demonstrations, and gained little or nothing on unseen tasks. By contrast, weight-based learning showed substantial gains on unseen tasks. The authors therefore point toward future weight-based continual learning that could teach models to use harnesses in new domains. But the reported comparison covers the models and methods tested in this paper; it does not establish how the pattern will hold in other settings.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 10, 2026 (2026-11-10).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-18T13:32:44.496Z. First observed: 2026-11-18T05:19:19.236Z. Retrieved: 2026-11-18T13:04:28.018Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The reported findings concern the evaluated setup: the abstract names Qwen3.5-9B and GLM-5.3 Flash and does not establish whether the same pattern holds for other models or settings. The abstract describes results on tasks; it does not establish performance in real-world applications.
Open question
What evidence would show that weight-based can teach models to drive harnesses in new domains?
Source-stated
The authors propose Continual-ARC, a lifelong stream of tasks from three ARC-AGI editions, designed to compare memory-based and weight-based systems on the same scale.
We propose Continual-ARC, a lifelong stream built from the three ARC-AGI editions
Source-stated
In the benchmark, tasks recur after dormant gaps with fresh instances, and the learner has a feedback budget for demonstrations, answers and harness actions.
The same tasks return after dormant gaps with fresh instances, while new tasks keep arriving.
Source-stated
The authors report that harness persistence generally did not improve memorization without explicit expert guidance; most harness methods performed below a demonstration-caching floor.
Persistence in the harness generally does not improve memorization unless provided with explicit expert guidance. Most harness-based methods still learn less than a floor that simply caches each task's demonstrations
Source-stated
The authors report little or no harness-based gain on unseen tasks, while weight-based learning showed substantial gains; they suggest future work could teach models to drive harnesses in new domains.
Harness-based methods gain little or nothing on unseen tasks, whereas weight-based learning shows substantial gains.
Source and contributor context
Lenton, Daniel, Arif, Yusha, Mahmood, Haris · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Continual learning
A model improving from new experience after deployment, the way a new employee learns on the job, without full retraining.
Agent harness
An agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 2572ac9d-9e66-4da4-803f-3262116804a0.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T13:32:44.496Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: dce89cd9a294e91fbcde3d7b45d9bb6f0a947e39582bdad6efbefcba814a77f0. Source text hash: fbfcaac6912feb0b257a3c84da86f33512308aa3fc11b3d3c3fb041f48c6e2b4.
Revision and source-check history
Version 1 · 2026-11-18T13:32:44.496Z
Continual-ARC: Measuring Skill Acquisition over a Working Lifetime
Authors Daniel Lenton, Yusha Arif and Haris Mahmood propose Continual-ARC, a for studying : whether a system can acquire skills as tasks recur over . Its stream combines tasks from three ARC-AGI editions. Tasks return after gaps with fresh instances, while new tasks arrive; learners can pay for demonstrations, submit answers or act inside an , with paid retries for wrong answers. The benchmark scores how much feedback budget each instance uses, then tests learners on familiar tasks and unseen ones. The authors say this lets memory-based and weight-based systems be compared on the same scale. In their experiments, persistence in an agent harness generally did not improve memorization without explicit expert guidance. Most harness-based methods learned less than a baseline that simply cached task demonstrations, and gained little or nothing on unseen tasks. By contrast, weight-based learning showed substantial gains on unseen tasks. The authors therefore point toward future weight-based continual learning that could teach models to use harnesses in new domains. But the reported comparison covers the models and methods tested in this paper; it does not establish how the pattern will hold in other settings.
Original source
cs.AI · Article
Source details
Original source date: Nov 10, 2026 (2026-11-10).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-18T13:32:44.496Z. First observed: 2026-11-18T05:19:19.236Z. Retrieved: 2026-11-18T13:04:28.018Z.
Limits
The reported findings concern the evaluated setup: the abstract names Qwen3.5-9B and GLM-5.3 Flash and does not establish whether the same pattern holds for other models or settings. The abstract describes results on tasks; it does not establish performance in real-world applications.
Open question
What evidence would show that weight-based can teach models to drive harnesses in new domains?
Source-stated
The authors propose Continual-ARC, a lifelong stream of tasks from three ARC-AGI editions, designed to compare memory-based and weight-based systems on the same scale.
We propose Continual-ARC, a lifelong stream built from the three ARC-AGI editions
Source-stated
In the benchmark, tasks recur after dormant gaps with fresh instances, and the learner has a feedback budget for demonstrations, answers and harness actions.
The same tasks return after dormant gaps with fresh instances, while new tasks keep arriving.
Source-stated
The authors report that harness persistence generally did not improve memorization without explicit expert guidance; most harness methods performed below a demonstration-caching floor.
Persistence in the harness generally does not improve memorization unless provided with explicit expert guidance. Most harness-based methods still learn less than a floor that simply caches each task's demonstrations
Source-stated
The authors report little or no harness-based gain on unseen tasks, while weight-based learning showed substantial gains; they suggest future work could teach models to drive harnesses in new domains.
Harness-based methods gain little or nothing on unseen tasks, whereas weight-based learning shows substantial gains.
Source and contributor context
Lenton, Daniel, Arif, Yusha, Mahmood, Haris · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Benchmark
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Continual learning
A model improving from new experience after deployment, the way a new employee learns on the job, without full retraining.
Agent harness
An agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
TIME
TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Editorial provenance
Version 1. Publication ID: 2572ac9d-9e66-4da4-803f-3262116804a0.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-18T13:32:44.496Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: dce89cd9a294e91fbcde3d7b45d9bb6f0a947e39582bdad6efbefcba814a77f0. Source text hash: fbfcaac6912feb0b257a3c84da86f33512308aa3fc11b3d3c3fb041f48c6e2b4.
uncertain · 2026-11-18T13:32:44.496Z · anthropic/claude-opus-5.5
Publication: 2572ac9d-9e66-4da4-803f-3262116804a0
attribution — pass: The reading correctly credits Daniel Lenton, Yusha Arif and Haris Mahmood with the paper. It keeps the benchmark design and results attributed to the authors ('the authors say', 'in their experiments').
Authors:Daniel Lenton, Yusha Arif, Haris Mahmood
summary — uncertain: The reading tracks the abstract faithfully but leaves central terms unexplained for a non-specialist. The terms are 'agent harness' (the main object being tested), 'memory-based' versus 'weight-based' systems, and ARC-AGI. A general reader cannot follow the core finding without knowing what a harness is or what learning in weights means. It also omits two details: the harness ran with frozen weights, and the weight-based method was Qwen3.5-9B trained to imitate GLM-5.3 Flash. That bears on how 'substantial gains' should be read. Smallest repair: add one plain-language gloss or tooltip for each of the three terms, and name the specific weight-based method.
We run an agent harness with frozen weights, with and without persistence, on Qwen3.5-9B and GLM-5.3 Flash, comparing them with two prompting floors, a program-induction method and Qwen3.5-9B's weights trained to imitate GLM-5.3 Flash.
date — pass: originalPublished is 2026-11-10, which matches the v1 submission date. Later revisions on Nov 11 and Nov 15 and the Nov 18 feed entry do not change the original date.
Submitted on 10 Nov 2026 (v1)
support — pass: Each claim is directly supported by the abstract. The forward-looking statement is attributed to the authors as a possibility, not stated as fact. The limitations correctly scope the results to the tested models and benchmark. The question follows from the authors' stated direction without presupposing the answer.
Harness-based methods gain little or nothing on unseen tasks, whereas weight-based learning shows substantial gains.
media — pass: The selected media is the verified arXiv abstract page for this paper, which is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model improving from new experience after deployment, the way a new employee learns on the job, without full retraining.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEAn agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model improving from new experience after deployment, the way a new employee learns on the job, without full retraining.
Definition referenceA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model improving from new experience after deployment, the way a new employee learns on the job, without full retraining.
Definition referenceTIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.
Explore TIMEAn agent harness is the system that lets an AI model act as an agent: it processes inputs, orchestrates tool calls and returns results. An evaluation harness runs evals end to end, including grading outputs.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA defined set of tests used to compare systems. Results depend on the tasks and measures selected.
Definition referenceA model improving from new experience after deployment, the way a new employee learns on the job, without full retraining.
Definition reference