{"id":"69204d7d1dd876d6b83ba7053f092dcf100a25d71d78fbc6bddda3e780aa59cc","title":"Roko's Plan R: AI Safety by ASICs","argument":"Roko proposes separating AI research and development organizations from AI deployment companies, so no single organization both develops powerful AI and owns an unbounded financial claim on its output. He argues this could reduce AI risks without banning AI or pausing research. In his disaster scenario, an AI company gives the human economy IOUs in exchange for goods and services, creating an incentive to push ahead quickly to sell more IOUs. If a misaligned AI takes over, it could make those IOUs unenforceable.\n\nIn the proposal, research organizations train and test models under restrictions, while deployment companies run AI services and sell products. After a model passes required evaluations, its design would be fixed in an ASIC, a chip built to run that particular model. Roko proposes that research organizations delete or encrypt their model copies and that high-spec compute outside their facilities use ASICs, which cannot train models or switch to another model.\n\nThe plan also calls for isolated research facilities, delayed data connections, scrutiny, and limits on researchers using their own organization’s models. Roko argues these measures could constrain rapid, unilateral development while allowing AI services and products to continue.","topicIds":["intelligence-and-discovery","institutions-and-trust"],"domain":"science","claimType":"attributed reading","limits":"The post presents a proposal and its author's expectations, not evidence that the controls would work or reduce AI risk. The post says an international agreement would be needed, but does not analyze the bargaining frictions it identifies. The post leaves open how AI research and development organizations might be financed, noting that debt financing needs further consideration. ","speaker":{"name":"LessWrong","roleAtSource":"Publisher"},"source":{"title":"Plan R: AI Safety by ASICs","publisher":"LessWrong","format":"Article","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","publishedAt":"2026-10-29T02:08:07.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Roko proposes separating AI research and development organizations from AI deployment companies to avoid one organization having both powerful AI-development capabilities and an unbounded claim on the resulting surplus.","excerpt":"there is no single org with both properties.","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Source-stated"},{"text":"Under the proposal, a model would be placed in a fixed-function ASIC after evaluation, and its original form would no longer run in the research organization.","excerpt":"they would burn it into an ASIC. This would be a fixed-function ASIC","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Source-stated"},{"text":"Roko argues that the separation could reduce AI risk while allowing AI products and services to continue; the post does not demonstrate that the proposed measures would achieve this outcome.","excerpt":"we can probably get rid of most of AI risk without doing anything other than separating out the bookkeeping, physical footprint and institutions","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Escape Meridian interpretation"},{"text":"The proposal relies on international coordination to enforce the separation globally.","excerpt":"An international agreement would be needed to enforce this system (call it \"Plan R\") globally","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","checkedAt":null,"items":[],"note":"Reading based on the publisher's curated RSS article text and declared contributor; the original page link was checked separately."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Could the proposed ASIC controls and international agreement reliably prevent a research organization from circumventing the separation between AI research and deployment?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Roko's Plan R: AI Safety by ASICs","summary":"Roko proposes separating AI research and development organizations from AI deployment companies, so no single organization both develops powerful AI and owns an unbounded financial claim on its output. He argues this could reduce AI risks without banning AI or pausing research. In his disaster scenario, an AI company gives the human economy IOUs in exchange for goods and services, creating an incentive to push ahead quickly to sell more IOUs. If a misaligned AI takes over, it could make those IOUs unenforceable.\n\nIn the proposal, research organizations train and test models under restrictions, while deployment companies run AI services and sell products. After a model passes required evaluations, its design would be fixed in an ASIC, a chip built to run that particular model. Roko proposes that research organizations delete or encrypt their model copies and that high-spec compute outside their facilities use ASICs, which cannot train models or switch to another model.\n\nThe plan also calls for isolated research facilities, delayed data connections, scrutiny, and limits on researchers using their own organization’s models. Roko argues these measures could constrain rapid, unilateral development while allowing AI services and products to continue.","question":"Could the proposed ASIC controls and international agreement reliably prevent a research organization from circumventing the separation between AI research and deployment?","claims":[{"text":"Roko proposes separating AI research and development organizations from AI deployment companies to avoid one organization having both powerful AI-development capabilities and an unbounded claim on the resulting surplus.","excerpt":"there is no single org with both properties.","support":"source-stated"},{"text":"Under the proposal, a model would be placed in a fixed-function ASIC after evaluation, and its original form would no longer run in the research organization.","excerpt":"they would burn it into an ASIC. This would be a fixed-function ASIC","support":"source-stated"},{"text":"Roko argues that the separation could reduce AI risk while allowing AI products and services to continue; the post does not demonstrate that the proposed measures would achieve this outcome.","excerpt":"we can probably get rid of most of AI risk without doing anything other than separating out the bookkeeping, physical footprint and institutions","support":"interpretation"},{"text":"The proposal relies on international coordination to enforce the separation globally.","excerpt":"An international agreement would be needed to enforce this system (call it \"Plan R\") globally","support":"source-stated"}],"limitations":["The post presents a proposal and its author's expectations, not evidence that the controls would work or reduce AI risk.","The post says an international agreement would be needed, but does not analyze the bargaining frictions it identifies.","The post leaves open how AI research and development organizations might be financed, noting that debt financing needs further consideration. "],"mediaId":"8fae36eb141a9ccabbbbb8fe0ed4e4177af1fc03c5a826b5bd44088e363afaa4","correctionReason":null,"id":"69204d7d1dd876d6b83ba7053f092dcf100a25d71d78fbc6bddda3e780aa59cc","publicationId":"e82e78e6-4a51-49f3-83dc-85cc561595f8","version":1,"sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","sourceTitle":"Plan R: AI Safety by ASICs","publisher":"LessWrong","publishedAt":"2026-10-29T02:08:07.000Z","originalPublished":{"value":"2026-10-29T02:08:07.000Z","precision":"instant","basis":"feed_entry"},"firstObservedAt":"2026-10-29T02:12:43.983Z","createdAt":"2026-11-14T06:34:49.674Z","historical":true,"entityIds":["lesswrong"],"topicIds":["intelligence-and-discovery","institutions-and-trust"],"media":[{"id":"8fae36eb141a9ccabbbbb8fe0ed4e4177af1fc03c5a826b5bd44088e363afaa4","kind":"article","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-29T03:00:51.094Z","method":"registered_feed_entry_and_head","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","checkedAt":null,"items":[],"note":"Reading based on the publisher's curated RSS article text and declared contributor; the original page link was checked separately."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v5","mediaId":"8fae36eb141a9ccabbbbb8fe0ed4e4177af1fc03c5a826b5bd44088e363afaa4","confidence":0.97,"provider":"typesafe","model":"jev-1.13.0","inputHash":"5b0fe7993913a751869765382a8e9397f0a17026a2e5b157c334ac2cc96a710e","decidedAt":"2026-10-29T03:00:52.538Z"},"provenance":{"textHash":"bc7c43ef082296014ba13baef5e1d2bd5af87734b635e659854d7901e7b638d4","observationId":"1932780b0236557b02bc8e8824a6a35e12444ad72ab41abd55644bb0ab72eea0","retrievedAt":"2026-10-29T02:12:34.740Z","kind":"feed-text","preparer":{"model":"gpt-6-luna","runId":"216a4317-f660-4db3-8708-7359006bcaf2"},"reviewer":{"model":"gpt-6-luna","runId":"a9d2fa43-a028-4b58-851e-027dfba6a532"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-14T06:34:49.674Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":null},"canonicalUrl":"https://escapemeridian.com/readings/roko-s-plan-r-ai-safety-by-asics-69204d7d","formats":{"html":"/readings/roko-s-plan-r-ai-safety-by-asics-69204d7d","markdown":"/readings/roko-s-plan-r-ai-safety-by-asics-69204d7d.md","json":"/readings/roko-s-plan-r-ai-safety-by-asics-69204d7d.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-ai-alignment","name":"AI alignment","explanation":"Research and practice aimed at making AI systems pursue the goals their builders and users intend.","referenceUrl":"https://en.wikipedia.org/wiki/AI_alignment"}],"entities":[{"id":"lesswrong","name":"LessWrong","kind":"organization","context":"LessWrong publishes community essays and discussions and provides a curated RSS feed.","referenceUrls":["https://www.lesswrong.com/","https://www.lesswrong.com/feed.xml?view=curated-rss"],"path":"/entities/lesswrong","json":"/entities/lesswrong.json","markdown":"/entities/lesswrong.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"69204d7d1dd876d6b83ba7053f092dcf100a25d71d78fbc6bddda3e780aa59cc","revision":"e82e78e6-4a51-49f3-83dc-85cc561595f8","revisionKind":"publication-id","title":"Roko's Plan R: AI Safety by ASICs","url":"https://escapemeridian.com/readings/roko-s-plan-r-ai-safety-by-asics-69204d7d"},"source":{"type":"source","id":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","revision":"bc7c43ef082296014ba13baef5e1d2bd5af87734b635e659854d7901e7b638d4","revisionKind":"sha256-source-text","title":"Plan R: AI Safety by ASICs","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"69204d7d1dd876d6b83ba7053f092dcf100a25d71d78fbc6bddda3e780aa59cc","title":"Roko's Plan R: AI Safety by ASICs","argument":"Roko proposes separating AI research and development organizations from AI deployment companies, so no single organization both develops powerful AI and owns an unbounded financial claim on its output. He argues this could reduce AI risks without banning AI or pausing research. In his disaster scenario, an AI company gives the human economy IOUs in exchange for goods and services, creating an incentive to push ahead quickly to sell more IOUs. If a misaligned AI takes over, it could make those IOUs unenforceable.\n\nIn the proposal, research organizations train and test models under restrictions, while deployment companies run AI services and sell products. After a model passes required evaluations, its design would be fixed in an ASIC, a chip built to run that particular model. Roko proposes that research organizations delete or encrypt their model copies and that high-spec compute outside their facilities use ASICs, which cannot train models or switch to another model.\n\nThe plan also calls for isolated research facilities, delayed data connections, scrutiny, and limits on researchers using their own organization’s models. Roko argues these measures could constrain rapid, unilateral development while allowing AI services and products to continue.","topicIds":["intelligence-and-discovery","institutions-and-trust"],"domain":"science","claimType":"attributed reading","limits":"The post presents a proposal and its author's expectations, not evidence that the controls would work or reduce AI risk. The post says an international agreement would be needed, but does not analyze the bargaining frictions it identifies. The post leaves open how AI research and development organizations might be financed, noting that debt financing needs further consideration. ","speaker":{"name":"LessWrong","roleAtSource":"Publisher"},"source":{"title":"Plan R: AI Safety by ASICs","publisher":"LessWrong","format":"Article","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","publishedAt":"2026-10-29T02:08:07.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Roko proposes separating AI research and development organizations from AI deployment companies to avoid one organization having both powerful AI-development capabilities and an unbounded claim on the resulting surplus.","excerpt":"there is no single org with both properties.","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Source-stated"},{"text":"Under the proposal, a model would be placed in a fixed-function ASIC after evaluation, and its original form would no longer run in the research organization.","excerpt":"they would burn it into an ASIC. This would be a fixed-function ASIC","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Source-stated"},{"text":"Roko argues that the separation could reduce AI risk while allowing AI products and services to continue; the post does not demonstrate that the proposed measures would achieve this outcome.","excerpt":"we can probably get rid of most of AI risk without doing anything other than separating out the bookkeeping, physical footprint and institutions","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Escape Meridian interpretation"},{"text":"The proposal relies on international coordination to enforce the separation globally.","excerpt":"An international agreement would be needed to enforce this system (call it \"Plan R\") globally","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","checkedAt":null,"items":[],"note":"Reading based on the publisher's curated RSS article text and declared contributor; the original page link was checked separately."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"Could the proposed ASIC controls and international agreement reliably prevent a research organization from circumventing the separation between AI research and deployment?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Roko's Plan R: AI Safety by ASICs","summary":"Roko proposes separating AI research and development organizations from AI deployment companies, so no single organization both develops powerful AI and owns an unbounded financial claim on its output. He argues this could reduce AI risks without banning AI or pausing research. In his disaster scenario, an AI company gives the human economy IOUs in exchange for goods and services, creating an incentive to push ahead quickly to sell more IOUs. If a misaligned AI takes over, it could make those IOUs unenforceable.\n\nIn the proposal, research organizations train and test models under restrictions, while deployment companies run AI services and sell products. After a model passes required evaluations, its design would be fixed in an ASIC, a chip built to run that particular model. Roko proposes that research organizations delete or encrypt their model copies and that high-spec compute outside their facilities use ASICs, which cannot train models or switch to another model.\n\nThe plan also calls for isolated research facilities, delayed data connections, scrutiny, and limits on researchers using their own organization’s models. Roko argues these measures could constrain rapid, unilateral development while allowing AI services and products to continue.","question":"Could the proposed ASIC controls and international agreement reliably prevent a research organization from circumventing the separation between AI research and deployment?","claims":[{"text":"Roko proposes separating AI research and development organizations from AI deployment companies to avoid one organization having both powerful AI-development capabilities and an unbounded claim on the resulting surplus.","excerpt":"there is no single org with both properties.","support":"source-stated"},{"text":"Under the proposal, a model would be placed in a fixed-function ASIC after evaluation, and its original form would no longer run in the research organization.","excerpt":"they would burn it into an ASIC. This would be a fixed-function ASIC","support":"source-stated"},{"text":"Roko argues that the separation could reduce AI risk while allowing AI products and services to continue; the post does not demonstrate that the proposed measures would achieve this outcome.","excerpt":"we can probably get rid of most of AI risk without doing anything other than separating out the bookkeeping, physical footprint and institutions","support":"interpretation"},{"text":"The proposal relies on international coordination to enforce the separation globally.","excerpt":"An international agreement would be needed to enforce this system (call it \"Plan R\") globally","support":"source-stated"}],"limitations":["The post presents a proposal and its author's expectations, not evidence that the controls would work or reduce AI risk.","The post says an international agreement would be needed, but does not analyze the bargaining frictions it identifies.","The post leaves open how AI research and development organizations might be financed, noting that debt financing needs further consideration. "],"mediaId":"8fae36eb141a9ccabbbbb8fe0ed4e4177af1fc03c5a826b5bd44088e363afaa4","correctionReason":null,"id":"69204d7d1dd876d6b83ba7053f092dcf100a25d71d78fbc6bddda3e780aa59cc","publicationId":"e82e78e6-4a51-49f3-83dc-85cc561595f8","version":1,"sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","sourceTitle":"Plan R: AI Safety by ASICs","publisher":"LessWrong","publishedAt":"2026-10-29T02:08:07.000Z","originalPublished":{"value":"2026-10-29T02:08:07.000Z","precision":"instant","basis":"feed_entry"},"firstObservedAt":"2026-10-29T02:12:43.983Z","createdAt":"2026-11-14T06:34:49.674Z","historical":true,"entityIds":["lesswrong"],"topicIds":["intelligence-and-discovery","institutions-and-trust"],"media":[{"id":"8fae36eb141a9ccabbbbb8fe0ed4e4177af1fc03c5a826b5bd44088e363afaa4","kind":"article","url":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-29T03:00:51.094Z","method":"registered_feed_entry_and_head","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.lesswrong.com/posts/n8u3BfqFoGh4jnzpo/plan-r-ai-safety-by-asics","checkedAt":null,"items":[],"note":"Reading based on the publisher's curated RSS article text and declared contributor; the original page link was checked separately."},"editorialDecision":{"outcome":"accept","reasonCodes":["useful_perspective"],"rubricVersion":"stream-admission-v5","mediaId":"8fae36eb141a9ccabbbbb8fe0ed4e4177af1fc03c5a826b5bd44088e363afaa4","confidence":0.97,"provider":"typesafe","model":"jev-1.13.0","inputHash":"5b0fe7993913a751869765382a8e9397f0a17026a2e5b157c334ac2cc96a710e","decidedAt":"2026-10-29T03:00:52.538Z"},"provenance":{"textHash":"bc7c43ef082296014ba13baef5e1d2bd5af87734b635e659854d7901e7b638d4","observationId":"1932780b0236557b02bc8e8824a6a35e12444ad72ab41abd55644bb0ab72eea0","retrievedAt":"2026-10-29T02:12:34.740Z","kind":"feed-text","preparer":{"model":"gpt-6-luna","runId":"216a4317-f660-4db3-8708-7359006bcaf2"},"reviewer":{"model":"gpt-6-luna","runId":"a9d2fa43-a028-4b58-851e-027dfba6a532"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-14T06:34:49.674Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":null}}],"audits":[{"publicationId":"e82e78e6-4a51-49f3-83dc-85cc561595f8","verdict":"uncertain","findings":[{"dimension":"attribution","status":"uncertain","explanation":"Roko is correctly named from the feed-supplied author, and the proposal is his. However, the draft calls the IOU/takeover story 'his disaster scenario.' The post presents it as a typical existing scenario, citing Daniel Kokotajlo as an example. Smallest repair: 'In a typical AI disaster scenario Roko describes (citing Daniel Kokotajlo as an example)...'","excerpt":"What a typical AI disaster scenario currently looks like (e.g. those by Daniel Kokotajlo)"},{"dimension":"summary","status":"uncertain","explanation":"The reading is clear, explains ASIC and IOUs, and covers the structure accurately. It softens the author's force: Roko claims separation can 'probably get rid of most of AI risk,' but the draft says 'reduce AI risks.' It also drops reasoning central to the argument: the nuclear/bioweapon contrast showing why (B) drives risk; the scenario's endpoint ('Everyone dies'); the claim that no rogue AI could capture a compute overhang; and the 'AI Safety by skeuomorphism' rationale of making AI more human-like. Repair: restore 'most of AI risk' and add one sentence on the skeuomorphism/nuclear rationale.","excerpt":"Nuclear technology and bioweapon technology both approximate (A) and they are mostly okay because without (B), there isn't an incentive for people controlling them to push their luck on safety."},{"dimension":"date","status":"pass","explanation":"originalPublished 2026-10-29T02:08:07Z comes from the feed entry and matches feedPublished, shortly before first observation. No earlier publication is indicated. The draft makes no conflicting date claim.","excerpt":null},{"dimension":"support","status":"pass","explanation":"The claims are grounded in the source: the separation of the two kinds of organization; burning models into fixed-function ASICs and deleting or encrypting them; the inability of ASICs to train or switch models; airgapped facilities with delayed connections; anti-dogfooding; and the need for an international agreement. The limitations on bargaining frictions and debt financing match the text. The interpretation claim properly notes the post does not demonstrate its outcome. The question follows from the evidence.","excerpt":"ASICs cannot compute gradients, so they cannot train. They cannot change what model they are running, so they cannot be taken over by rogue AIs."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original LessWrong post, matching the source URL.","excerpt":null}],"createdAt":"2026-11-14T06:34:49.674Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}