{"id":"eeef7ca24568c0fe592aa98201faccea099ec6507dbd9476c8bea56310f28743","title":"METR reports an unexpected slowdown in experienced open-source development","argument":"METR published this study on July 10, 2025. In a randomized study of experienced developers working on issues in open-source projects they knew well, METR reports that developers took longer when allowed to use AI tools. The finding ran against what participants expected: they thought AI would speed them up, and still believed it had helped after completing the work. METR says the result is about this particular setting, not evidence that AI fails to help most developers, other kinds of work, or developers using AI in the future. The study’s practical point is that benchmark scores and reports of helpfulness do not, by themselves, tell us whether AI speeds up demanding software work. METR argues that the differences between those kinds of evidence matter for understanding AI’s possible role in accelerating AI research and development, which it says could carry risks for oversight and safeguards. METR offers several possible explanations for the conflicting evidence, including that the methods measure different kinds of tasks. Which explanation best accounts for the gap remains unresolved.","topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 24, 2026; originally published July 10, 2025. METR says the developers and repositories may not represent software development work generally, and acknowledges possible participation and other sampling biases. The result concerns experienced developers working on familiar open-source projects; METR says it does not establish effects in other settings or with future AI systems.","speaker":{"name":"METR","roleAtSource":"Publisher"},"source":{"title":"Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity","publisher":"METR","format":"Article","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","publishedAt":"2025-07-10T12:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"METR reports that developers took longer on issues when allowed to use AI tools.","excerpt":"When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts.","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR says participants expected AI to speed them up and continued to believe it had after the study.","excerpt":"developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%.","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR says the study does not establish that AI fails to speed up most software developers or people in other domains.","excerpt":"We do not claim that our developers or repositories represent a majority or plurality of software development work","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR presents different task types and measures as one possible reason the study, benchmarks, and anecdotes point in different directions.","excerpt":"Hypothesis 3: Complementary evidence for different settings\nAll three methodologies are basically correct, but are measuring subsets of the “real” task distribution that are more or less challenging for models","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR says that substantial AI-driven acceleration of developers in this setting could signal broader acceleration of AI R&D, with possible risks to oversight and safeguards.","excerpt":"If AI systems are able to substantially speed up developers in our setting, this could signal rapid acceleration of AI R&D progress generally, which may in turn lead to proliferation risks, breakdowns in safeguards and oversight, or excess centralization of power.","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What explains the gap between this study’s slowdown and reports of AI helping with substantial software tasks: differences in tasks and methods, or limitations of one or more forms of evidence?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"METR reports an unexpected slowdown in experienced open-source development","summary":"METR published this study on July 10, 2025. In a randomized study of experienced developers working on issues in open-source projects they knew well, METR reports that developers took longer when allowed to use AI tools. The finding ran against what participants expected: they thought AI would speed them up, and still believed it had helped after completing the work. METR says the result is about this particular setting, not evidence that AI fails to help most developers, other kinds of work, or developers using AI in the future. The study’s practical point is that benchmark scores and reports of helpfulness do not, by themselves, tell us whether AI speeds up demanding software work. METR argues that the differences between those kinds of evidence matter for understanding AI’s possible role in accelerating AI research and development, which it says could carry risks for oversight and safeguards. METR offers several possible explanations for the conflicting evidence, including that the methods measure different kinds of tasks. Which explanation best accounts for the gap remains unresolved.","question":"What explains the gap between this study’s slowdown and reports of AI helping with substantial software tasks: differences in tasks and methods, or limitations of one or more forms of evidence?","claims":[{"text":"METR reports that developers took longer on issues when allowed to use AI tools.","excerpt":"When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts.","support":"source-stated"},{"text":"METR says participants expected AI to speed them up and continued to believe it had after the study.","excerpt":"developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%.","support":"source-stated"},{"text":"METR says the study does not establish that AI fails to speed up most software developers or people in other domains.","excerpt":"We do not claim that our developers or repositories represent a majority or plurality of software development work","support":"source-stated"},{"text":"METR presents different task types and measures as one possible reason the study, benchmarks, and anecdotes point in different directions.","excerpt":"Hypothesis 3: Complementary evidence for different settings\nAll three methodologies are basically correct, but are measuring subsets of the “real” task distribution that are more or less challenging for models","support":"source-stated"},{"text":"METR says that substantial AI-driven acceleration of developers in this setting could signal broader acceleration of AI R&D, with possible risks to oversight and safeguards.","excerpt":"If AI systems are able to substantially speed up developers in our setting, this could signal rapid acceleration of AI R&D progress generally, which may in turn lead to proliferation risks, breakdowns in safeguards and oversight, or excess centralization of power.","support":"source-stated"}],"limitations":["Text as retrieved on October 24, 2026; originally published July 10, 2025.","METR says the developers and repositories may not represent software development work generally, and acknowledges possible participation and other sampling biases.","The result concerns experienced developers working on familiar open-source projects; METR says it does not establish effects in other settings or with future AI systems."],"mediaId":null,"correctionReason":null,"id":"eeef7ca24568c0fe592aa98201faccea099ec6507dbd9476c8bea56310f28743","publicationId":"961c3d45-e26c-15d9-fcea-f0f5fa50a7b2","version":1,"sourceUrl":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","sourceTitle":"Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity","publisher":"METR","publishedAt":"2025-07-10T12:00:00.000Z","originalPublished":{"value":"2025-07-10T12:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:27:32.214Z","createdAt":"2026-11-13T04:33:24.893Z","historical":true,"entityIds":["metr"],"topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"8f13d046f03fd2c866cd104d320d5251430cf7582e1e74853dd44fc116608724","rubricVersion":"unrecorded","provider":"vercel","model":"typesafe-ai/jev","reasonCodes":["bridge_admitted_by_model"],"confidence":0.82,"decidedAt":"2026-10-24T02:03:45.186Z"},"provenance":{"textHash":"78a7743c1dda83f49b4d646c134243e5e6c38bed0eb7aa33d124a37e3971f7df","observationId":"7845b8690717cc94647936889f4dcd5080e0dbc10be8940a8bbeb99c7dd518b0","retrievedAt":"2026-10-24T02:05:35.833Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"fb22fca1-bb9c-07d1-9d1c-9e9671391965"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"8ecd8217-218f-c092-a652-efe3ea23c436"}},"audit":{"status":"passed","reviewedAt":"2026-11-13T04:28:55.340Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/metr-reports-an-unexpected-slowdown-in-experienced-open-source-eeef7ca2","formats":{"html":"/readings/metr-reports-an-unexpected-slowdown-in-experienced-open-source-eeef7ca2","markdown":"/readings/metr-reports-an-unexpected-slowdown-in-experienced-open-source-eeef7ca2.md","json":"/readings/metr-reports-an-unexpected-slowdown-in-experienced-open-source-eeef7ca2.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"metr","name":"METR","kind":"organization","context":"METR studies what AI systems can do and how reliably they complete tasks, including work with human developers.","referenceUrls":["https://metr.org/"],"path":"/entities/metr","json":"/entities/metr.json","markdown":"/entities/metr.md"}]},"researchQuestions":[{"id":"ai-and-work","title":"When does AI become a dependable colleague?","question":"When does an AI become a dependable colleague?","url":"/questions/ai-and-work","edition":"product-management-read-2026-11-04"},{"id":"ai-work-quality-and-rework","title":"Does AI reduce the effort needed to finish work well?","question":"Does AI reduce the total effort needed to produce work that meets the required standard?","url":"/questions/ai-work-quality-and-rework","edition":"product-management-read-2026-11-04"},{"id":"ai-delegation-reliability","title":"When can work be delegated without checking every step?","question":"When can work be delegated without checking every step?","url":"/questions/ai-delegation-reliability","edition":"product-management-read-2026-11-04"},{"id":"ai-deployment-returns","title":"When is an AI deployment worth its total cost?","question":"When does a useful AI deployment become worth its total cost?","url":"/questions/ai-deployment-returns","edition":"product-management-read-2026-11-04"}],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"eeef7ca24568c0fe592aa98201faccea099ec6507dbd9476c8bea56310f28743","revision":"961c3d45-e26c-15d9-fcea-f0f5fa50a7b2","revisionKind":"publication-id","title":"METR reports an unexpected slowdown in experienced open-source development","url":"https://escapemeridian.com/readings/metr-reports-an-unexpected-slowdown-in-experienced-open-source-eeef7ca2"},"source":{"type":"source","id":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","revision":"78a7743c1dda83f49b4d646c134243e5e6c38bed0eb7aa33d124a37e3971f7df","revisionKind":"sha256-source-text","title":"Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/"},"status":"available","items":[{"id":"reading-citation:amodei-fast-diffusion:3b9176ef7c6a259fbe18b442f8cbb8d3e1e78cab9accec9cd2caf5a1342776b3:evidence.1.url","from":{"type":"reading","id":"amodei-fast-diffusion","revision":"3b9176ef7c6a259fbe18b442f8cbb8d3e1e78cab9accec9cd2caf5a1342776b3","revisionKind":"sha256-record","title":"Dario Amodei: adoption can move fast and still take work","url":"https://escapemeridian.com/readings/amodei-fast-diffusion"},"to":{"type":"source","id":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","revision":null,"revisionKind":null,"title":"Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/"},"relationship":"references","provenance":{"kind":"recorded-citation","recordId":"3b9176ef7c6a259fbe18b442f8cbb8d3e1e78cab9accec9cd2caf5a1342776b3","field":"evidence.1.url","recordedAt":null,"author":{"id":null,"name":"Escape Meridian","kind":"editorial"},"citedUrl":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","assertedRelationship":"qualifies"}}],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"eeef7ca24568c0fe592aa98201faccea099ec6507dbd9476c8bea56310f28743","title":"METR reports an unexpected slowdown in experienced open-source development","argument":"METR published this study on July 10, 2025. In a randomized study of experienced developers working on issues in open-source projects they knew well, METR reports that developers took longer when allowed to use AI tools. The finding ran against what participants expected: they thought AI would speed them up, and still believed it had helped after completing the work. METR says the result is about this particular setting, not evidence that AI fails to help most developers, other kinds of work, or developers using AI in the future. The study’s practical point is that benchmark scores and reports of helpfulness do not, by themselves, tell us whether AI speeds up demanding software work. METR argues that the differences between those kinds of evidence matter for understanding AI’s possible role in accelerating AI research and development, which it says could carry risks for oversight and safeguards. METR offers several possible explanations for the conflicting evidence, including that the methods measure different kinds of tasks. Which explanation best accounts for the gap remains unresolved.","topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 24, 2026; originally published July 10, 2025. METR says the developers and repositories may not represent software development work generally, and acknowledges possible participation and other sampling biases. The result concerns experienced developers working on familiar open-source projects; METR says it does not establish effects in other settings or with future AI systems.","speaker":{"name":"METR","roleAtSource":"Publisher"},"source":{"title":"Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity","publisher":"METR","format":"Article","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","publishedAt":"2025-07-10T12:00:00.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"METR reports that developers took longer on issues when allowed to use AI tools.","excerpt":"When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts.","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR says participants expected AI to speed them up and continued to believe it had after the study.","excerpt":"developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%.","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR says the study does not establish that AI fails to speed up most software developers or people in other domains.","excerpt":"We do not claim that our developers or repositories represent a majority or plurality of software development work","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR presents different task types and measures as one possible reason the study, benchmarks, and anecdotes point in different directions.","excerpt":"Hypothesis 3: Complementary evidence for different settings\nAll three methodologies are basically correct, but are measuring subsets of the “real” task distribution that are more or less challenging for models","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"},{"text":"METR says that substantial AI-driven acceleration of developers in this setting could signal broader acceleration of AI R&D, with possible risks to oversight and safeguards.","excerpt":"If AI systems are able to substantially speed up developers in our setting, this could signal rapid acceleration of AI R&D progress generally, which may in turn lead to proliferation risks, breakdowns in safeguards and oversight, or excess centralization of power.","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What explains the gap between this study’s slowdown and reports of AI helping with substantial software tasks: differences in tasks and methods, or limitations of one or more forms of evidence?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"METR reports an unexpected slowdown in experienced open-source development","summary":"METR published this study on July 10, 2025. In a randomized study of experienced developers working on issues in open-source projects they knew well, METR reports that developers took longer when allowed to use AI tools. The finding ran against what participants expected: they thought AI would speed them up, and still believed it had helped after completing the work. METR says the result is about this particular setting, not evidence that AI fails to help most developers, other kinds of work, or developers using AI in the future. The study’s practical point is that benchmark scores and reports of helpfulness do not, by themselves, tell us whether AI speeds up demanding software work. METR argues that the differences between those kinds of evidence matter for understanding AI’s possible role in accelerating AI research and development, which it says could carry risks for oversight and safeguards. METR offers several possible explanations for the conflicting evidence, including that the methods measure different kinds of tasks. Which explanation best accounts for the gap remains unresolved.","question":"What explains the gap between this study’s slowdown and reports of AI helping with substantial software tasks: differences in tasks and methods, or limitations of one or more forms of evidence?","claims":[{"text":"METR reports that developers took longer on issues when allowed to use AI tools.","excerpt":"When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts.","support":"source-stated"},{"text":"METR says participants expected AI to speed them up and continued to believe it had after the study.","excerpt":"developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%.","support":"source-stated"},{"text":"METR says the study does not establish that AI fails to speed up most software developers or people in other domains.","excerpt":"We do not claim that our developers or repositories represent a majority or plurality of software development work","support":"source-stated"},{"text":"METR presents different task types and measures as one possible reason the study, benchmarks, and anecdotes point in different directions.","excerpt":"Hypothesis 3: Complementary evidence for different settings\nAll three methodologies are basically correct, but are measuring subsets of the “real” task distribution that are more or less challenging for models","support":"source-stated"},{"text":"METR says that substantial AI-driven acceleration of developers in this setting could signal broader acceleration of AI R&D, with possible risks to oversight and safeguards.","excerpt":"If AI systems are able to substantially speed up developers in our setting, this could signal rapid acceleration of AI R&D progress generally, which may in turn lead to proliferation risks, breakdowns in safeguards and oversight, or excess centralization of power.","support":"source-stated"}],"limitations":["Text as retrieved on October 24, 2026; originally published July 10, 2025.","METR says the developers and repositories may not represent software development work generally, and acknowledges possible participation and other sampling biases.","The result concerns experienced developers working on familiar open-source projects; METR says it does not establish effects in other settings or with future AI systems."],"mediaId":null,"correctionReason":null,"id":"eeef7ca24568c0fe592aa98201faccea099ec6507dbd9476c8bea56310f28743","publicationId":"961c3d45-e26c-15d9-fcea-f0f5fa50a7b2","version":1,"sourceUrl":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","sourceTitle":"Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity","publisher":"METR","publishedAt":"2025-07-10T12:00:00.000Z","originalPublished":{"value":"2025-07-10T12:00:00.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-18T03:27:32.214Z","createdAt":"2026-11-13T04:33:24.893Z","historical":true,"entityIds":["metr"],"topicIds":["work-and-business--jobs-and-occupations","job-displacement-and-creation"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"8f13d046f03fd2c866cd104d320d5251430cf7582e1e74853dd44fc116608724","rubricVersion":"unrecorded","provider":"vercel","model":"typesafe-ai/jev","reasonCodes":["bridge_admitted_by_model"],"confidence":0.82,"decidedAt":"2026-10-24T02:03:45.186Z"},"provenance":{"textHash":"78a7743c1dda83f49b4d646c134243e5e6c38bed0eb7aa33d124a37e3971f7df","observationId":"7845b8690717cc94647936889f4dcd5080e0dbc10be8940a8bbeb99c7dd518b0","retrievedAt":"2026-10-24T02:05:35.833Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"fb22fca1-bb9c-07d1-9d1c-9e9671391965"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"8ecd8217-218f-c092-a652-efe3ea23c436"}},"audit":{"status":"passed","reviewedAt":"2026-11-13T04:28:55.340Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"961c3d45-e26c-15d9-fcea-f0f5fa50a7b2","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes the study, its findings, its stated non-claims and its risk argument to METR, the publisher of this single blog post. Source claims are kept as METR's reports rather than presented as established fact.","excerpt":"When developers are allowed to use AI tools, they take 19% longer to complete issues"},{"dimension":"summary","status":"pass","explanation":"The reading leads with the actor and the randomized slowdown finding. It explains the perception gap, METR's scope limits and why METR says the result matters for AI R&D acceleration. A non-specialist can follow it, and benchmark is defined on the page. Omitting the 19%, 24% and 20% figures from the summary is a stylistic choice; the claims carry them. The question reflects METR's own unresolved hypotheses and is not a leading claim.","excerpt":"Reconciling these different sources of evidence is difficult but important, and in part it depends on what question we’re trying to answer."},{"dimension":"date","status":"pass","explanation":"July 10, 2025 matches the UTC day of originalPublished, and the evidence is a publisher page, so no archive disclosure is needed. The page's later out-of-date notice and February 2026 update look like additions after publication, but the reading neither uses nor relies on them. METR's forward-looking risk statement is kept as a conditional claim.","excerpt":"Update: In February 2026, we published new data on the productivity impact of late-2025 AI tools."},{"dimension":"support","status":"pass","explanation":"Each claim matches its verbatim excerpt. The summary's points about the scope limits, conflicting evidence, oversight risks and unresolved explanations are all supported by the source. No unsupported benefits, costs or forecasts are introduced.","excerpt":"We do not claim that our developers or repositories represent a majority or plurality of software development work"},{"dimension":"media","status":"pass","explanation":"No media was selected, and none of the supplied candidates were verified, so leaving media empty is appropriate.","excerpt":null}],"createdAt":"2026-11-13T04:28:55.340Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}