Roko's Plan R: AI Safety by ASICs

Roko's Plan R: AI Safety by ASICs

Roko proposes separating AI research and development organizations from AI deployment companies, so no single organization both develops powerful AI and owns an unbounded financial claim on its output. He argues this could reduce AI risks without banning AI or pausing research. In his disaster scenario, an AI company gives the human economy IOUs in exchange for goods and services, creating an incentive to push ahead quickly to sell more IOUs. If a misaligned AI takes over, it could make those IOUs unenforceable. In the proposal, research organizations train and test models under restrictions, while deployment companies run AI services and sell products. After a model passes required evaluations, its design would be fixed in an ASIC, a chip built to run that particular model. Roko proposes that research organizations delete or encrypt their model copies and that high-spec compute outside their facilities use ASICs, which cannot train models or switch to another model. The plan also calls for isolated research facilities, delayed data connections, scrutiny, and limits on researchers using their own organization’s models. Roko argues these measures could constrain rapid, unilateral development while allowing AI services and products to continue.

Publication: e82e78e6-4a51-49f3-83dc-85cc561595f8

attribution — uncertain: Roko is correctly named from the feed-supplied author, and the proposal is his. However, the draft calls the IOU/takeover story 'his disaster scenario.' The post presents it as a typical existing scenario, citing Daniel Kokotajlo as an example. Smallest repair: 'In a typical AI disaster scenario Roko describes (citing Daniel Kokotajlo as an example)...'

What a typical AI disaster scenario currently looks like (e.g. those by Daniel Kokotajlo)

summary — uncertain: The reading is clear, explains ASIC and IOUs, and covers the structure accurately. It softens the author's force: Roko claims separation can 'probably get rid of most of AI risk,' but the draft says 'reduce AI risks.' It also drops reasoning central to the argument: the nuclear/bioweapon contrast showing why (B) drives risk; the scenario's endpoint ('Everyone dies'); the claim that no rogue AI could capture a compute overhang; and the 'AI Safety by skeuomorphism' rationale of making AI more human-like. Repair: restore 'most of AI risk' and add one sentence on the skeuomorphism/nuclear rationale.

Nuclear technology and bioweapon technology both approximate (A) and they are mostly okay because without (B), there isn't an incentive for people controlling them to push their luck on safety.

date — pass: originalPublished 2026-10-29T02:08:07Z comes from the feed entry and matches feedPublished, shortly before first observation. No earlier publication is indicated. The draft makes no conflicting date claim.

support — pass: The claims are grounded in the source: the separation of the two kinds of organization; burning models into fixed-function ASICs and deleting or encrypting them; the inability of ASICs to train or switch models; airgapped facilities with delayed connections; anti-dogfooding; and the need for an international agreement. The limitations on bargaining frictions and debt financing match the text. The interpretation claim properly notes the post does not demonstrate its outcome. The question follows from the evidence.

ASICs cannot compute gradients, so they cannot train. They cannot change what model they are running, so they cannot be taken over by rogue AIs.

media — pass: The selected media is the verified direct link to the original LessWrong post, matching the source URL.