Ajeya Cotra · Aug 20, 2024
Luca Righetti argues in Planned Obsolescence for harder dangerous-capability tests
Markdown · JSON · Original source
Luca Righetti’s essay was published on August 20, 2024. He argues that current tests of AI’s dangerous biological capabilities can be too easy to justify costly precautions, yet passing them may still leave unclear whether an AI can help someone create a biological weapon. Knowledge tests do not establish that a model can troubleshoot protocols: biological work can require adapting detailed procedures to a particular laboratory. Even stronger tests may not show that a layperson can act on AI advice or overcome practical barriers outside a professional lab. Righetti describes a one-way problem: failure on a test may suggest safety, while passing still may not establish danger. He proposes working backward from a hypothetical “gold standard” test: a in which non-experts attempt to create a relatively harmless virus, with AI mentors or internet access, using a secure garage-like lab. He says such a test would be costly and might be impractical or unethical, but could give decision-makers a stronger basis for costly measures. His conclusion is that difficult, realistic tests should be designed in advance, rather than made harder without a clear plan. Whether a safe, ethical test could establish the capabilities that would justify those measures remains unresolved.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
Ajeya Cotra · Article
Source details
Original source date: Aug 20, 2024 (2024-08-20T15:42:19+00:00).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-17T11:33:27.197Z. First observed: 2026-10-28T20:34:00.543Z. Retrieved: 2026-10-28T20:51:33.658Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Text as retrieved on October 28, 2026; originally published August 20, 2024. The proposed gold-standard test is hypothetical; the essay does not report that it was conducted. Righetti notes that the Future House study compared models without tools against human evaluators who could use tools, a limitation that may affect the reported comparison.
Open question
What safe, realistic test could show whether AI meaningfully helps non-experts overcome the practical barriers to creating a biological weapon?
Source-stated
Righetti argues that passing a dangerous-capability test may not establish that an AI can help a layperson create a biological weapon.
if it succeeds, it’s still not clear whether it’s actually dangerous
Source-stated
Righetti says biological work involves adapting protocols to a specific laboratory, not just recalling facts.
It involves designing detailed, step-by-step plans (known as “protocols”) and tailoring them to a specific laboratory environment.
Source-stated
Righetti proposes working backward from a hypothetical test in which non-experts try to create a relatively harmless virus, with AI mentors or internet access.
Half the people would have AI mentors and the other half can only look stuff up on the internet.
Source-stated
Righetti argues that difficult, realistic tests should be designed in advance to help justify costly measures.
we need to agree in advance on some tests that are hard enough and realistic enough that they clearly justify action.
Source-stated
Righetti says the hypothetical test could be too impractical or unethical, but argues that designing hard tests should begin now.
even if this exact test turns out to be too impractical (or unethical) to be worth it, we need to agree in advance on some tests
Source and contributor context
Ajeya Cotra · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Randomized controlled trial
A study that randomly assigns people to groups so that differences in outcomes can be traced to the treatment.
Editorial provenance
Version 1. Publication ID: db14d3ff-0461-e5e3-3412-505d6b276f21.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-17T11:28:42.326Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 6a9b656d15082dba0fa53e051bd6efba9392901aecfc8c41d4064aac98bd7bc0. Source text hash: 2dccc6022b4fa1a33bef0574b596676b61660d2f785126769fcf915b38eeb49b.
Revision and source-check history
Version 1 · 2026-11-17T11:33:27.197Z
Luca Righetti argues in Planned Obsolescence for harder dangerous-capability tests
Luca Righetti’s essay was published on August 20, 2024. He argues that current tests of AI’s dangerous biological capabilities can be too easy to justify costly precautions, yet passing them may still leave unclear whether an AI can help someone create a biological weapon. Knowledge tests do not establish that a model can troubleshoot protocols: biological work can require adapting detailed procedures to a particular laboratory. Even stronger tests may not show that a layperson can act on AI advice or overcome practical barriers outside a professional lab. Righetti describes a one-way problem: failure on a test may suggest safety, while passing still may not establish danger. He proposes working backward from a hypothetical “gold standard” test: a in which non-experts attempt to create a relatively harmless virus, with AI mentors or internet access, using a secure garage-like lab. He says such a test would be costly and might be impractical or unethical, but could give decision-makers a stronger basis for costly measures. His conclusion is that difficult, realistic tests should be designed in advance, rather than made harder without a clear plan. Whether a safe, ethical test could establish the capabilities that would justify those measures remains unresolved.
Original source
Ajeya Cotra · Article
Source details
Original source date: Aug 20, 2024 (2024-08-20T15:42:19+00:00).
Date basis: Date stated on the publisher’s page.
Added to Escape Meridian: 2026-11-17T11:33:27.197Z. First observed: 2026-10-28T20:34:00.543Z. Retrieved: 2026-10-28T20:51:33.658Z.
Limits
Text as retrieved on October 28, 2026; originally published August 20, 2024. The proposed gold-standard test is hypothetical; the essay does not report that it was conducted. Righetti notes that the Future House study compared models without tools against human evaluators who could use tools, a limitation that may affect the reported comparison.
Open question
What safe, realistic test could show whether AI meaningfully helps non-experts overcome the practical barriers to creating a biological weapon?
Source-stated
Righetti argues that passing a dangerous-capability test may not establish that an AI can help a layperson create a biological weapon.
if it succeeds, it’s still not clear whether it’s actually dangerous
Source-stated
Righetti says biological work involves adapting protocols to a specific laboratory, not just recalling facts.
It involves designing detailed, step-by-step plans (known as “protocols”) and tailoring them to a specific laboratory environment.
Source-stated
Righetti proposes working backward from a hypothetical test in which non-experts try to create a relatively harmless virus, with AI mentors or internet access.
Half the people would have AI mentors and the other half can only look stuff up on the internet.
Source-stated
Righetti argues that difficult, realistic tests should be designed in advance to help justify costly measures.
we need to agree in advance on some tests that are hard enough and realistic enough that they clearly justify action.
Source-stated
Righetti says the hypothetical test could be too impractical or unethical, but argues that designing hard tests should begin now.
even if this exact test turns out to be too impractical (or unethical) to be worth it, we need to agree in advance on some tests
Source and contributor context
Ajeya Cotra · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media retrieval status: unchecked.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Randomized controlled trial
A study that randomly assigns people to groups so that differences in outcomes can be traced to the treatment.
Editorial provenance
Version 1. Publication ID: db14d3ff-0461-e5e3-3412-505d6b276f21.
Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-17T11:28:42.326Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.
Observation: 6a9b656d15082dba0fa53e051bd6efba9392901aecfc8c41d4064aac98bd7bc0. Source text hash: 2dccc6022b4fa1a33bef0574b596676b61660d2f785126769fcf915b38eeb49b.
passed · 2026-11-17T11:28:42.326Z · anthropic/claude-opus-5.5
Publication: db14d3ff-0461-e5e3-3412-505d6b276f21
attribution — pass: The essay is credited to Luca Righetti, matching the byline. It appears on the Planned Obsolescence site (planned-obsolescence.org), edited by Ajeya Cotra. Every argument in the reading is attributed to Righetti.
Luca Righetti Aug 20, 2024
summary — pass: The reading captures the core argument: tests are one-directional, knowledge tests miss protocol troubleshooting and tacit skills, and we should work backward from a hypothetical gold-standard test. It also keeps the author's conclusion that such tests should be agreed in advance. Protocols are explained in plain language, and the randomized controlled trial term is defined on the page. The urgency the essay ties to possible 2025–2026 progress is only implied by 'begin now', but this does not prevent understanding. The question follows from the evidence.
All these tests have a weird one-directionality to them: If an AI fails, it’s probably safe; but if it succeeds, it’s still not clear whether it’s actually dangerous.
date — pass: The stated date, August 20, 2024, matches the dateline and the originalPublished instant (UTC). The modification timestamp is the same as publication. The page is a publisher page, so no archive disclosure is needed. There is no hindsight framing, and the essay's predictions are not presented as outcomes.
Aug 20, 2024
support — pass: Each claim is source-stated and backed by its excerpt. The limitations accurately note that the test is hypothetical and accurately describe the Future House tools caveat from the footnote. The reading implies no unsupported benefits or costs. Its statement that a costly gold-standard test could give decision-makers a stronger basis tracks the source.
I could actually imagine experts and decision-makers agreeing that if an AI passes this kind of test, then it poses massive risks
media — pass: No media is selected, and no verified media candidates were supplied. This is appropriate.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A study that randomly assigns people to groups so that differences in outcomes can be traced to the treatment.
Definition referenceA study that randomly assigns people to groups so that differences in outcomes can be traced to the treatment.
Definition reference