Escape Meridian

Ajeya Cotra · Aug 20, 2024

Luca Righetti argues in Planned Obsolescence for harder dangerous-capability tests

Markdown · JSON · Original source

Luca Righetti’s essay was published on August 20, 2024. He argues that current tests of AI’s dangerous biological capabilities can be too easy to justify costly precautions, yet passing them may still leave unclear whether an AI can help someone create a biological weapon. Knowledge tests do not establish that a model can troubleshoot protocols: biological work can require adapting detailed procedures to a particular laboratory. Even stronger tests may not show that a layperson can act on AI advice or overcome practical barriers outside a professional lab. Righetti describes a one-way problem: failure on a test may suggest safety, while passing still may not establish danger. He proposes working backward from a hypothetical “gold standard” test: a in which non-experts attempt to create a relatively harmless virus, with AI mentors or internet access, using a secure garage-like lab. He says such a test would be costly and might be impractical or unethical, but could give decision-makers a stronger basis for costly measures. His conclusion is that difficult, realistic tests should be designed in advance, rather than made harder without a clear plan. Whether a safe, ethical test could establish the capabilities that would justify those measures remains unresolved.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

Ajeya CotraArticle · planned-obsolescence.orgOpen original source

Original source

Ajeya Cotra · Article

Source details

Original source date: Aug 20, 2024 (2024-08-20T15:42:19+00:00).

Date basis: Date stated on the publisher’s page.

Added to Escape Meridian: 2026-11-17T11:33:27.197Z. First observed: 2026-10-28T20:34:00.543Z. Retrieved: 2026-10-28T20:51:33.658Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

Text as retrieved on October 28, 2026; originally published August 20, 2024. The proposed gold-standard test is hypothetical; the essay does not report that it was conducted. Righetti notes that the Future House study compared models without tools against human evaluators who could use tools, a limitation that may affect the reported comparison.

Open question

What safe, realistic test could show whether AI meaningfully helps non-experts overcome the practical barriers to creating a biological weapon?

Source-stated

Righetti argues that passing a dangerous-capability test may not establish that an AI can help a layperson create a biological weapon.

if it succeeds, it’s still not clear whether it’s actually dangerous

Source-stated

Righetti says biological work involves adapting protocols to a specific laboratory, not just recalling facts.

It involves designing detailed, step-by-step plans (known as “protocols”) and tailoring them to a specific laboratory environment.

Source-stated

Righetti proposes working backward from a hypothetical test in which non-experts try to create a relatively harmless virus, with AI mentors or internet access.

Half the people would have AI mentors and the other half can only look stuff up on the internet.

Source-stated

Righetti argues that difficult, realistic tests should be designed in advance to help justify costly measures.

we need to agree in advance on some tests that are hard enough and realistic enough that they clearly justify action.

Source-stated

Righetti says the hypothetical test could be too impractical or unethical, but argues that designing hard tests should begin now.

even if this exact test turns out to be too impractical (or unethical) to be worth it, we need to agree in advance on some tests

Source and contributor context

Ajeya Cotra · Publisher

The publisher is identified here. No individual author has been inferred.

Source media

Media retrieval status: unchecked.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Randomized controlled trial

A study that randomly assigns people to groups so that differences in outcomes can be traced to the treatment.

Editorial provenance

Version 1. Publication ID: db14d3ff-0461-e5e3-3412-505d6b276f21.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-17T11:28:42.326Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.

Observation: 6a9b656d15082dba0fa53e051bd6efba9392901aecfc8c41d4064aac98bd7bc0. Source text hash: 2dccc6022b4fa1a33bef0574b596676b61660d2f785126769fcf915b38eeb49b.

Revision and source-check history

Version 1 · 2026-11-17T11:33:27.197Z

Luca Righetti argues in Planned Obsolescence for harder dangerous-capability tests

Luca Righetti’s essay was published on August 20, 2024. He argues that current tests of AI’s dangerous biological capabilities can be too easy to justify costly precautions, yet passing them may still leave unclear whether an AI can help someone create a biological weapon. Knowledge tests do not establish that a model can troubleshoot protocols: biological work can require adapting detailed procedures to a particular laboratory. Even stronger tests may not show that a layperson can act on AI advice or overcome practical barriers outside a professional lab. Righetti describes a one-way problem: failure on a test may suggest safety, while passing still may not establish danger. He proposes working backward from a hypothetical “gold standard” test: a in which non-experts attempt to create a relatively harmless virus, with AI mentors or internet access, using a secure garage-like lab. He says such a test would be costly and might be impractical or unethical, but could give decision-makers a stronger basis for costly measures. His conclusion is that difficult, realistic tests should be designed in advance, rather than made harder without a clear plan. Whether a safe, ethical test could establish the capabilities that would justify those measures remains unresolved.

Original source

Ajeya Cotra · Article

Source details

Original source date: Aug 20, 2024 (2024-08-20T15:42:19+00:00).

Date basis: Date stated on the publisher’s page.

Added to Escape Meridian: 2026-11-17T11:33:27.197Z. First observed: 2026-10-28T20:34:00.543Z. Retrieved: 2026-10-28T20:51:33.658Z.

Limits

Text as retrieved on October 28, 2026; originally published August 20, 2024. The proposed gold-standard test is hypothetical; the essay does not report that it was conducted. Righetti notes that the Future House study compared models without tools against human evaluators who could use tools, a limitation that may affect the reported comparison.

Open question

What safe, realistic test could show whether AI meaningfully helps non-experts overcome the practical barriers to creating a biological weapon?

Source-stated

Righetti argues that passing a dangerous-capability test may not establish that an AI can help a layperson create a biological weapon.

if it succeeds, it’s still not clear whether it’s actually dangerous

Source-stated

Righetti says biological work involves adapting protocols to a specific laboratory, not just recalling facts.

It involves designing detailed, step-by-step plans (known as “protocols”) and tailoring them to a specific laboratory environment.

Source-stated

Righetti proposes working backward from a hypothetical test in which non-experts try to create a relatively harmless virus, with AI mentors or internet access.

Half the people would have AI mentors and the other half can only look stuff up on the internet.

Source-stated

Righetti argues that difficult, realistic tests should be designed in advance to help justify costly measures.

we need to agree in advance on some tests that are hard enough and realistic enough that they clearly justify action.

Source-stated

Righetti says the hypothetical test could be too impractical or unethical, but argues that designing hard tests should begin now.

even if this exact test turns out to be too impractical (or unethical) to be worth it, we need to agree in advance on some tests

Source and contributor context

Ajeya Cotra · Publisher

The publisher is identified here. No individual author has been inferred.

Source media

Media retrieval status: unchecked.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Randomized controlled trial

A study that randomly assigns people to groups so that differences in outcomes can be traced to the treatment.

Editorial provenance

Version 1. Publication ID: db14d3ff-0461-e5e3-3412-505d6b276f21.

Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-17T11:28:42.326Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: bridge_admitted_by_model, second_opinion_categorical.

Observation: 6a9b656d15082dba0fa53e051bd6efba9392901aecfc8c41d4064aac98bd7bc0. Source text hash: 2dccc6022b4fa1a33bef0574b596676b61660d2f785126769fcf915b38eeb49b.

passed · 2026-11-17T11:28:42.326Z · anthropic/claude-opus-5.5

Publication: db14d3ff-0461-e5e3-3412-505d6b276f21

attribution — pass: The essay is credited to Luca Righetti, matching the byline. It appears on the Planned Obsolescence site (planned-obsolescence.org), edited by Ajeya Cotra. Every argument in the reading is attributed to Righetti.

Luca Righetti Aug 20, 2024

summary — pass: The reading captures the core argument: tests are one-directional, knowledge tests miss protocol troubleshooting and tacit skills, and we should work backward from a hypothetical gold-standard test. It also keeps the author's conclusion that such tests should be agreed in advance. Protocols are explained in plain language, and the randomized controlled trial term is defined on the page. The urgency the essay ties to possible 2025–2026 progress is only implied by 'begin now', but this does not prevent understanding. The question follows from the evidence.

All these tests have a weird one-directionality to them: If an AI fails, it’s probably safe; but if it succeeds, it’s still not clear whether it’s actually dangerous.

date — pass: The stated date, August 20, 2024, matches the dateline and the originalPublished instant (UTC). The modification timestamp is the same as publication. The page is a publisher page, so no archive disclosure is needed. There is no hindsight framing, and the essay's predictions are not presented as outcomes.

Aug 20, 2024

support — pass: Each claim is source-stated and backed by its excerpt. The limitations accurately note that the test is hypothetical and accurately describe the Future House tools caveat from the footnote. The reading implies no unsupported benefits or costs. Its statement that a costly gold-standard test could give decision-makers a stronger basis tracks the source.

I could actually imagine experts and decision-makers agreeing that if an AI passes this kind of test, then it poses massive risks

media — pass: No media is selected, and no verified media candidates were supplied. This is appropriate.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.