{"id":"d0423bee09af99fcf1024e1260d1e5afbdfd7e3b824b58bfa0eee64f53a3cf63","title":"Toby Ord asks whether AI agents’ task success has a half-life","argument":"Toby Ord’s article was published on May 7, 2025. Building on work by Kwa et al., Ord argues that a constant rate of failure during each minute a human would take to do a task could explain how AI agents’ success rates fall as tasks get longer. This model implies an exponentially declining success rate, which Ord says would give each agent a half-life: a task-length measure of the point where its success rate is 50%. Ord’s proposed mechanism is that longer tasks involve more subtasks, and failing any one can sink the whole task. That model could help estimate success at different task lengths and understand why an agent that can complete one day’s work may not reliably complete two days. But Ord stresses that the pattern is only suggestive, and that the task suite may not represent performance elsewhere. He calls for comparisons of the model with alternatives, analysis of human and AI performance, and tests on other task suites. Whether the model generalises beyond this suite, and whether it fits the data better than alternatives, remains unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 27, 2026; originally published May 7, 2025. Ord says the proposed model is suggestive, not a precise description of agents’ failure rates. The article’s update describes later analysis and is not included as part of the original article’s argument.","speaker":{"name":"Toby Ord","roleAtSource":"Publisher"},"source":{"title":"Is there a Half-Life for the Success Rates of AI Agents?","publisher":"Toby Ord","format":"Article","url":"https://www.tobyord.com/writing/half-life","publishedAt":"2025-05-07T11:54:05+01:00","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"Toby Ord argues that a constant rate of failure per minute of human task time could explain the decline in AI agents’ success on longer tasks in Kwa et al.’s research-engineering task suite.","excerpt":"the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take to do the task.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"},{"text":"Ord says this model implies an exponentially declining success rate and a half-life for each agent, measured by the human time needed to complete a task.","excerpt":"This implies an exponentially declining success rate with the length of the task and that each agent could be characterised by its own half-life.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"},{"text":"Ord suggests that longer tasks may involve more subtasks, with failure at any one causing the overall task to fail.","excerpt":"they involve increasingly large sets of subtasks where failing any one fails the task.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"},{"text":"Ord says the model’s generality is unknown and calls for statistical comparisons and tests on other task suites.","excerpt":"And of course it is also important to know how much any of this generalises to other suites of tasks.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.tobyord.com/writing/half-life","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does an exponential decay model fit METR’s data compared with alternative models, and does it generalise to other task suites?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Toby Ord asks whether AI agents’ task success has a half-life","summary":"Toby Ord’s article was published on May 7, 2025. Building on work by Kwa et al., Ord argues that a constant rate of failure during each minute a human would take to do a task could explain how AI agents’ success rates fall as tasks get longer. This model implies an exponentially declining success rate, which Ord says would give each agent a half-life: a task-length measure of the point where its success rate is 50%. Ord’s proposed mechanism is that longer tasks involve more subtasks, and failing any one can sink the whole task. That model could help estimate success at different task lengths and understand why an agent that can complete one day’s work may not reliably complete two days. But Ord stresses that the pattern is only suggestive, and that the task suite may not represent performance elsewhere. He calls for comparisons of the model with alternatives, analysis of human and AI performance, and tests on other task suites. Whether the model generalises beyond this suite, and whether it fits the data better than alternatives, remains unknown.","question":"How well does an exponential decay model fit METR’s data compared with alternative models, and does it generalise to other task suites?","claims":[{"text":"Toby Ord argues that a constant rate of failure per minute of human task time could explain the decline in AI agents’ success on longer tasks in Kwa et al.’s research-engineering task suite.","excerpt":"the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take to do the task.","support":"source-stated"},{"text":"Ord says this model implies an exponentially declining success rate and a half-life for each agent, measured by the human time needed to complete a task.","excerpt":"This implies an exponentially declining success rate with the length of the task and that each agent could be characterised by its own half-life.","support":"source-stated"},{"text":"Ord suggests that longer tasks may involve more subtasks, with failure at any one causing the overall task to fail.","excerpt":"they involve increasingly large sets of subtasks where failing any one fails the task.","support":"source-stated"},{"text":"Ord says the model’s generality is unknown and calls for statistical comparisons and tests on other task suites.","excerpt":"And of course it is also important to know how much any of this generalises to other suites of tasks.","support":"source-stated"}],"limitations":["Text as retrieved on October 27, 2026; originally published May 7, 2025.","Ord says the proposed model is suggestive, not a precise description of agents’ failure rates. The article’s update describes later analysis and is not included as part of the original article’s argument."],"mediaId":null,"correctionReason":null,"id":"d0423bee09af99fcf1024e1260d1e5afbdfd7e3b824b58bfa0eee64f53a3cf63","publicationId":"a6e539d8-66af-cf2b-38c3-bae0e50f87cb","version":1,"sourceUrl":"https://www.tobyord.com/writing/half-life","sourceTitle":"Is there a Half-Life for the Success Rates of AI Agents?","publisher":"Toby Ord","publishedAt":"2025-05-07T11:54:05+01:00","originalPublished":{"value":"2025-05-07T11:54:05+01:00","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-10-27T16:23:39.483Z","createdAt":"2026-11-14T04:53:23.863Z","historical":true,"entityIds":[],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.tobyord.com/writing/half-life","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"840dd28e596193448f3ecf9d3a4e056027e0ecdd744c8b8201a70562d43382f3","rubricVersion":"unrecorded","provider":"typesafe","model":"jev-1.13.0","reasonCodes":["bridge_admitted_by_model"],"confidence":0.83,"decidedAt":"2026-10-27T16:23:38.922Z"},"provenance":{"textHash":"721a51ad309e45ef2540154a083edf0514cfcdfbb3ae1bacc69e7d63f5debb3f","observationId":"45bee58a68613b4ecbd033fbf302cad34fb7de206bedf69aff6b858857d41e73","retrievedAt":"2026-10-27T16:25:35.516Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"2c6608f1-078d-1eea-b1d3-ed2b6f55e5b1"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"573aa527-a599-af9f-eb41-94897ef03c88"}},"audit":{"status":"passed","reviewedAt":"2026-11-14T04:48:42.576Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/toby-ord-asks-whether-ai-agents-task-success-has-a-half-life-d0423bee","formats":{"html":"/readings/toby-ord-asks-whether-ai-agents-task-success-has-a-half-life-d0423bee","markdown":"/readings/toby-ord-asks-whether-ai-agents-task-success-has-a-half-life-d0423bee.md","json":"/readings/toby-ord-asks-whether-ai-agents-task-success-has-a-half-life-d0423bee.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-ai-agent","name":"AI agent","explanation":"An AI system that chooses and carries out actions toward a goal, often using software tools.","referenceUrl":"https://www.ibm.com/think/topics/ai-agents"}],"entities":[{"id":"metr","name":"METR","kind":"organization","context":"METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.","referenceUrls":["https://metr.org/"],"path":"/entities/metr","json":"/entities/metr.json","markdown":"/entities/metr.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"d0423bee09af99fcf1024e1260d1e5afbdfd7e3b824b58bfa0eee64f53a3cf63","revision":"a6e539d8-66af-cf2b-38c3-bae0e50f87cb","revisionKind":"publication-id","title":"Toby Ord asks whether AI agents’ task success has a half-life","url":"https://escapemeridian.com/readings/toby-ord-asks-whether-ai-agents-task-success-has-a-half-life-d0423bee"},"source":{"type":"source","id":"https://www.tobyord.com/writing/half-life","revision":"721a51ad309e45ef2540154a083edf0514cfcdfbb3ae1bacc69e7d63f5debb3f","revisionKind":"sha256-source-text","title":"Is there a Half-Life for the Success Rates of AI Agents?","url":"https://www.tobyord.com/writing/half-life"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"d0423bee09af99fcf1024e1260d1e5afbdfd7e3b824b58bfa0eee64f53a3cf63","title":"Toby Ord asks whether AI agents’ task success has a half-life","argument":"Toby Ord’s article was published on May 7, 2025. Building on work by Kwa et al., Ord argues that a constant rate of failure during each minute a human would take to do a task could explain how AI agents’ success rates fall as tasks get longer. This model implies an exponentially declining success rate, which Ord says would give each agent a half-life: a task-length measure of the point where its success rate is 50%. Ord’s proposed mechanism is that longer tasks involve more subtasks, and failing any one can sink the whole task. That model could help estimate success at different task lengths and understand why an agent that can complete one day’s work may not reliably complete two days. But Ord stresses that the pattern is only suggestive, and that the task suite may not represent performance elsewhere. He calls for comparisons of the model with alternatives, analysis of human and AI performance, and tests on other task suites. Whether the model generalises beyond this suite, and whether it fits the data better than alternatives, remains unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 27, 2026; originally published May 7, 2025. Ord says the proposed model is suggestive, not a precise description of agents’ failure rates. The article’s update describes later analysis and is not included as part of the original article’s argument.","speaker":{"name":"Toby Ord","roleAtSource":"Publisher"},"source":{"title":"Is there a Half-Life for the Success Rates of AI Agents?","publisher":"Toby Ord","format":"Article","url":"https://www.tobyord.com/writing/half-life","publishedAt":"2025-05-07T11:54:05+01:00","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"Toby Ord argues that a constant rate of failure per minute of human task time could explain the decline in AI agents’ success on longer tasks in Kwa et al.’s research-engineering task suite.","excerpt":"the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take to do the task.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"},{"text":"Ord says this model implies an exponentially declining success rate and a half-life for each agent, measured by the human time needed to complete a task.","excerpt":"This implies an exponentially declining success rate with the length of the task and that each agent could be characterised by its own half-life.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"},{"text":"Ord suggests that longer tasks may involve more subtasks, with failure at any one causing the overall task to fail.","excerpt":"they involve increasingly large sets of subtasks where failing any one fails the task.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"},{"text":"Ord says the model’s generality is unknown and calls for statistical comparisons and tests on other task suites.","excerpt":"And of course it is also important to know how much any of this generalises to other suites of tasks.","url":"https://www.tobyord.com/writing/half-life","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.tobyord.com/writing/half-life","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well does an exponential decay model fit METR’s data compared with alternative models, and does it generalise to other task suites?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Toby Ord asks whether AI agents’ task success has a half-life","summary":"Toby Ord’s article was published on May 7, 2025. Building on work by Kwa et al., Ord argues that a constant rate of failure during each minute a human would take to do a task could explain how AI agents’ success rates fall as tasks get longer. This model implies an exponentially declining success rate, which Ord says would give each agent a half-life: a task-length measure of the point where its success rate is 50%. Ord’s proposed mechanism is that longer tasks involve more subtasks, and failing any one can sink the whole task. That model could help estimate success at different task lengths and understand why an agent that can complete one day’s work may not reliably complete two days. But Ord stresses that the pattern is only suggestive, and that the task suite may not represent performance elsewhere. He calls for comparisons of the model with alternatives, analysis of human and AI performance, and tests on other task suites. Whether the model generalises beyond this suite, and whether it fits the data better than alternatives, remains unknown.","question":"How well does an exponential decay model fit METR’s data compared with alternative models, and does it generalise to other task suites?","claims":[{"text":"Toby Ord argues that a constant rate of failure per minute of human task time could explain the decline in AI agents’ success on longer tasks in Kwa et al.’s research-engineering task suite.","excerpt":"the performance of AI agents on longer-duration tasks can be explained by an extremely simple mathematical model — a constant rate of failing during each minute a human would take to do the task.","support":"source-stated"},{"text":"Ord says this model implies an exponentially declining success rate and a half-life for each agent, measured by the human time needed to complete a task.","excerpt":"This implies an exponentially declining success rate with the length of the task and that each agent could be characterised by its own half-life.","support":"source-stated"},{"text":"Ord suggests that longer tasks may involve more subtasks, with failure at any one causing the overall task to fail.","excerpt":"they involve increasingly large sets of subtasks where failing any one fails the task.","support":"source-stated"},{"text":"Ord says the model’s generality is unknown and calls for statistical comparisons and tests on other task suites.","excerpt":"And of course it is also important to know how much any of this generalises to other suites of tasks.","support":"source-stated"}],"limitations":["Text as retrieved on October 27, 2026; originally published May 7, 2025.","Ord says the proposed model is suggestive, not a precise description of agents’ failure rates. The article’s update describes later analysis and is not included as part of the original article’s argument."],"mediaId":null,"correctionReason":null,"id":"d0423bee09af99fcf1024e1260d1e5afbdfd7e3b824b58bfa0eee64f53a3cf63","publicationId":"a6e539d8-66af-cf2b-38c3-bae0e50f87cb","version":1,"sourceUrl":"https://www.tobyord.com/writing/half-life","sourceTitle":"Is there a Half-Life for the Success Rates of AI Agents?","publisher":"Toby Ord","publishedAt":"2025-05-07T11:54:05+01:00","originalPublished":{"value":"2025-05-07T11:54:05+01:00","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-10-27T16:23:39.483Z","createdAt":"2026-11-14T04:53:23.863Z","historical":true,"entityIds":[],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.tobyord.com/writing/half-life","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"840dd28e596193448f3ecf9d3a4e056027e0ecdd744c8b8201a70562d43382f3","rubricVersion":"unrecorded","provider":"typesafe","model":"jev-1.13.0","reasonCodes":["bridge_admitted_by_model"],"confidence":0.83,"decidedAt":"2026-10-27T16:23:38.922Z"},"provenance":{"textHash":"721a51ad309e45ef2540154a083edf0514cfcdfbb3ae1bacc69e7d63f5debb3f","observationId":"45bee58a68613b4ecbd033fbf302cad34fb7de206bedf69aff6b858857d41e73","retrievedAt":"2026-10-27T16:25:35.516Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"2c6608f1-078d-1eea-b1d3-ed2b6f55e5b1"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"573aa527-a599-af9f-eb41-94897ef03c88"}},"audit":{"status":"passed","reviewedAt":"2026-11-14T04:48:42.576Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"a6e539d8-66af-cf2b-38c3-bae0e50f87cb","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"This is one essay by Toby Ord on his own site. The reading attributes the model and argument to Ord, and the underlying data to Kwa et al. Ord's claims stay framed as his arguments, not as established fact.","excerpt":"Building on the recent empirical work of Kwa et al. (2025), I show that within their suite of research-engineering tasks"},{"dimension":"summary","status":"pass","explanation":"The reading leads with Ord's argument and explains the constant failure rate, the half-life (the task length at which success is 50%) and the subtask mechanism in plain language. It keeps the one-day versus two-day illustration and Ord's hedges. The question names METR without introducing it, which is a minor gap; the core reading remains understandable. The question also follows directly from Ord's further-work section.","excerpt":"Ideally one would conduct a formal statistical analysis on how well an exponential decay curve fits METRs data compared to the log-logistic they use."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2025-05-07T11:54+01:00, which falls on May 7, 2025 in UTC and matches the dateline. The page was edited after publication with a Feb 2026 update. The reading explicitly excludes that update, and every cited excerpt comes from the original body. There is no hindsight framing, and the page is a publisher page, not an archive copy.","excerpt":"May 7, 2025Toby Ord"},{"dimension":"support","status":"pass","explanation":"Each claim matches its excerpt, and all cited text is plainly part of the original essay, not the later update. Two further statements are also supported in the source: that the pattern is only suggestive, and the one-day versus two-day illustration. The reading adds no invented benefits or costs.","excerpt":"So far these results and analysis are merely suggestive of a constant hazard rate."},{"dimension":"media","status":"pass","explanation":"No media was selected and no verified candidates were supplied, so nothing needs checking.","excerpt":null}],"createdAt":"2026-11-14T04:48:42.576Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}