{"id":"607e571864c3b6803e040ee04630e117861dde6d2c1da761a53147d8bb7ab085","title":"Scott Alexander argues in Astral Codex Ten that neuralese recurrence raises a taboo question","argument":"Scott Alexander’s piece, published September 24, 2026, asks whether OpenAI’s use of recurrence crosses a safety boundary around neuralese. He distinguishes chain-of-thought, where a model writes intermediate reasoning as text, from neuralese, where internal representations loop through additional processing without being written out. Alexander says Astra uses loops to add simulated layers, but this is not true neuralese, which would let an AI reason without chain-of-thought; he says nobody has invented that technology in real life.\n\nAlexander argues that loops may let a model do more processing between visible reasoning steps, potentially allowing dangerous planning to go unobserved. Repeated use of the same layers can help, he says, but does not provide the same capabilities gain as adding real layers. His central argument is that a taboo needs a clear boundary, or exceptions can weaken it. He suggests tabooing loops, setting a maximum number of real or simulated layers, or agreeing on the components of true neuralese. He concludes that AI company leaders could formalize a taboo, which he argues might be maintainable through voluntary commitments. The scale of any safety risk and whether companies will adopt such a taboo remain unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 23, 2026; originally published September 24, 2026. Alexander describes possible safety risks and proposes possible taboos; the piece does not establish the scale of any resulting risk or whether companies will adopt them.","speaker":{"name":"Scott Alexander","roleAtSource":"Publisher"},"source":{"title":"The Specter Of Neuralese","publisher":"Scott Alexander","format":"Article","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","publishedAt":"2026-09-24T22:12:39.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Scott Alexander’s piece is dated September 24, 2026.","excerpt":"Sep 24, 2026","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"},{"text":"Alexander distinguishes recurrence that adds simulated layers from true neuralese, which avoids chain-of-thought entirely; he says nobody has invented the latter in real life.","excerpt":"The dangerous AIs in AI 2027 are (3), but so far nobody has invented this in real life.","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"},{"text":"Alexander argues that a taboo needs a specific boundary, and offers three possible approaches: tabooing loops, setting a maximum number of layers, or agreeing on the components of true neuralese.","excerpt":"First, contra Pachocki, it might be worth instituting a taboo on looping (adding “simulated layers”) at all","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"},{"text":"Alexander argues that AI company leaders could formalize a taboo through voluntary commitments.","excerpt":"one more useful thing they could do would be to formalize this taboo, so we know exactly what it is we’re trying to stay away from.","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.astralcodexten.com/p/the-specter-of-neuralese","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What boundary, if any, would AI companies agree to use for a taboo on recurrence or neuralese, and how would they define it?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Scott Alexander argues in Astral Codex Ten that neuralese recurrence raises a taboo question","summary":"Scott Alexander’s piece, published September 24, 2026, asks whether OpenAI’s use of recurrence crosses a safety boundary around neuralese. He distinguishes chain-of-thought, where a model writes intermediate reasoning as text, from neuralese, where internal representations loop through additional processing without being written out. Alexander says Astra uses loops to add simulated layers, but this is not true neuralese, which would let an AI reason without chain-of-thought; he says nobody has invented that technology in real life.\n\nAlexander argues that loops may let a model do more processing between visible reasoning steps, potentially allowing dangerous planning to go unobserved. Repeated use of the same layers can help, he says, but does not provide the same capabilities gain as adding real layers. His central argument is that a taboo needs a clear boundary, or exceptions can weaken it. He suggests tabooing loops, setting a maximum number of real or simulated layers, or agreeing on the components of true neuralese. He concludes that AI company leaders could formalize a taboo, which he argues might be maintainable through voluntary commitments. The scale of any safety risk and whether companies will adopt such a taboo remain unknown.","question":"What boundary, if any, would AI companies agree to use for a taboo on recurrence or neuralese, and how would they define it?","claims":[{"text":"Scott Alexander’s piece is dated September 24, 2026.","excerpt":"Sep 24, 2026","support":"source-stated"},{"text":"Alexander distinguishes recurrence that adds simulated layers from true neuralese, which avoids chain-of-thought entirely; he says nobody has invented the latter in real life.","excerpt":"The dangerous AIs in AI 2027 are (3), but so far nobody has invented this in real life.","support":"source-stated"},{"text":"Alexander argues that a taboo needs a specific boundary, and offers three possible approaches: tabooing loops, setting a maximum number of layers, or agreeing on the components of true neuralese.","excerpt":"First, contra Pachocki, it might be worth instituting a taboo on looping (adding “simulated layers”) at all","support":"source-stated"},{"text":"Alexander argues that AI company leaders could formalize a taboo through voluntary commitments.","excerpt":"one more useful thing they could do would be to formalize this taboo, so we know exactly what it is we’re trying to stay away from.","support":"source-stated"}],"limitations":["Text as retrieved on October 23, 2026; originally published September 24, 2026.","Alexander describes possible safety risks and proposes possible taboos; the piece does not establish the scale of any resulting risk or whether companies will adopt them."],"mediaId":null,"correctionReason":null,"id":"607e571864c3b6803e040ee04630e117861dde6d2c1da761a53147d8bb7ab085","publicationId":"853460ae-2ef8-310c-fa8c-74cb826ed931","version":1,"sourceUrl":"https://www.astralcodexten.com/p/the-specter-of-neuralese","sourceTitle":"The Specter Of Neuralese","publisher":"Scott Alexander","publishedAt":"2026-09-24T22:12:39.000Z","originalPublished":{"value":"2026-09-24T22:12:39.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-23T23:05:57.734Z","createdAt":"2026-11-05T14:33:24.563Z","historical":true,"entityIds":["scott-alexander"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.astralcodexten.com/p/the-specter-of-neuralese","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"fec56d1650120a9d5125c46be40fdeeac91504549f007dca0b88ddf07e93a12c","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-23T23:05:55.108Z"},"provenance":{"textHash":"fafc68b668078328adaed0d28e255f141fb6e040802e81da6d732b7083349920","observationId":"9a216f7bb70de3350d2b446a9478d92a17225c9a53553dc430111ab11f92dfe2","retrievedAt":"2026-10-23T23:07:45.376Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"fd97aaaf-ecac-0390-3f07-ac4d2af7aba8"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"4459ad41-e416-9088-be3c-42148dbc4131"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-05T14:28:26.533Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/scott-alexander-argues-in-astral-codex-ten-that-neuralese-607e5718","formats":{"html":"/readings/scott-alexander-argues-in-astral-codex-ten-that-neuralese-607e5718","markdown":"/readings/scott-alexander-argues-in-astral-codex-ten-that-neuralese-607e5718.md","json":"/readings/scott-alexander-argues-in-astral-codex-ten-that-neuralese-607e5718.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-chain-of-thought","name":"Chain of thought","explanation":"The step-by-step working a model writes out before giving its final answer.","referenceUrl":"https://developers.google.com/machine-learning/glossary#chain-of-thought-prompting"}],"entities":[{"id":"scott-alexander","name":"Scott Alexander","kind":"person","context":"Scott Alexander authored essays published in the Slate Star Codex archive.","referenceUrls":["https://slatestarcodex.com/archives/","https://slatestarcodex.com/about/"],"path":"/entities/scott-alexander","json":"/entities/scott-alexander.json","markdown":"/entities/scott-alexander.md"},{"id":"openai","name":"OpenAI","kind":"organization","context":"OpenAI develops AI models and products and publishes research, release announcements and evaluations.","referenceUrls":["https://openai.com/news/"],"path":"/entities/openai","json":"/entities/openai.json","markdown":"/entities/openai.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"607e571864c3b6803e040ee04630e117861dde6d2c1da761a53147d8bb7ab085","revision":"853460ae-2ef8-310c-fa8c-74cb826ed931","revisionKind":"publication-id","title":"Scott Alexander argues in Astral Codex Ten that neuralese recurrence raises a taboo question","url":"https://escapemeridian.com/readings/scott-alexander-argues-in-astral-codex-ten-that-neuralese-607e5718"},"source":{"type":"source","id":"https://www.astralcodexten.com/p/the-specter-of-neuralese","revision":"fafc68b668078328adaed0d28e255f141fb6e040802e81da6d732b7083349920","revisionKind":"sha256-source-text","title":"The Specter Of Neuralese","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"607e571864c3b6803e040ee04630e117861dde6d2c1da761a53147d8bb7ab085","title":"Scott Alexander argues in Astral Codex Ten that neuralese recurrence raises a taboo question","argument":"Scott Alexander’s piece, published September 24, 2026, asks whether OpenAI’s use of recurrence crosses a safety boundary around neuralese. He distinguishes chain-of-thought, where a model writes intermediate reasoning as text, from neuralese, where internal representations loop through additional processing without being written out. Alexander says Astra uses loops to add simulated layers, but this is not true neuralese, which would let an AI reason without chain-of-thought; he says nobody has invented that technology in real life.\n\nAlexander argues that loops may let a model do more processing between visible reasoning steps, potentially allowing dangerous planning to go unobserved. Repeated use of the same layers can help, he says, but does not provide the same capabilities gain as adding real layers. His central argument is that a taboo needs a clear boundary, or exceptions can weaken it. He suggests tabooing loops, setting a maximum number of real or simulated layers, or agreeing on the components of true neuralese. He concludes that AI company leaders could formalize a taboo, which he argues might be maintainable through voluntary commitments. The scale of any safety risk and whether companies will adopt such a taboo remain unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 23, 2026; originally published September 24, 2026. Alexander describes possible safety risks and proposes possible taboos; the piece does not establish the scale of any resulting risk or whether companies will adopt them.","speaker":{"name":"Scott Alexander","roleAtSource":"Publisher"},"source":{"title":"The Specter Of Neuralese","publisher":"Scott Alexander","format":"Article","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","publishedAt":"2026-09-24T22:12:39.000Z","dateBasis":"Date supplied by the publisher’s feed."},"evidence":[{"text":"Scott Alexander’s piece is dated September 24, 2026.","excerpt":"Sep 24, 2026","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"},{"text":"Alexander distinguishes recurrence that adds simulated layers from true neuralese, which avoids chain-of-thought entirely; he says nobody has invented the latter in real life.","excerpt":"The dangerous AIs in AI 2027 are (3), but so far nobody has invented this in real life.","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"},{"text":"Alexander argues that a taboo needs a specific boundary, and offers three possible approaches: tabooing loops, setting a maximum number of layers, or agreeing on the components of true neuralese.","excerpt":"First, contra Pachocki, it might be worth instituting a taboo on looping (adding “simulated layers”) at all","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"},{"text":"Alexander argues that AI company leaders could formalize a taboo through voluntary commitments.","excerpt":"one more useful thing they could do would be to formalize this taboo, so we know exactly what it is we’re trying to stay away from.","url":"https://www.astralcodexten.com/p/the-specter-of-neuralese","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.astralcodexten.com/p/the-specter-of-neuralese","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What boundary, if any, would AI companies agree to use for a taboo on recurrence or neuralese, and how would they define it?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Scott Alexander argues in Astral Codex Ten that neuralese recurrence raises a taboo question","summary":"Scott Alexander’s piece, published September 24, 2026, asks whether OpenAI’s use of recurrence crosses a safety boundary around neuralese. He distinguishes chain-of-thought, where a model writes intermediate reasoning as text, from neuralese, where internal representations loop through additional processing without being written out. Alexander says Astra uses loops to add simulated layers, but this is not true neuralese, which would let an AI reason without chain-of-thought; he says nobody has invented that technology in real life.\n\nAlexander argues that loops may let a model do more processing between visible reasoning steps, potentially allowing dangerous planning to go unobserved. Repeated use of the same layers can help, he says, but does not provide the same capabilities gain as adding real layers. His central argument is that a taboo needs a clear boundary, or exceptions can weaken it. He suggests tabooing loops, setting a maximum number of real or simulated layers, or agreeing on the components of true neuralese. He concludes that AI company leaders could formalize a taboo, which he argues might be maintainable through voluntary commitments. The scale of any safety risk and whether companies will adopt such a taboo remain unknown.","question":"What boundary, if any, would AI companies agree to use for a taboo on recurrence or neuralese, and how would they define it?","claims":[{"text":"Scott Alexander’s piece is dated September 24, 2026.","excerpt":"Sep 24, 2026","support":"source-stated"},{"text":"Alexander distinguishes recurrence that adds simulated layers from true neuralese, which avoids chain-of-thought entirely; he says nobody has invented the latter in real life.","excerpt":"The dangerous AIs in AI 2027 are (3), but so far nobody has invented this in real life.","support":"source-stated"},{"text":"Alexander argues that a taboo needs a specific boundary, and offers three possible approaches: tabooing loops, setting a maximum number of layers, or agreeing on the components of true neuralese.","excerpt":"First, contra Pachocki, it might be worth instituting a taboo on looping (adding “simulated layers”) at all","support":"source-stated"},{"text":"Alexander argues that AI company leaders could formalize a taboo through voluntary commitments.","excerpt":"one more useful thing they could do would be to formalize this taboo, so we know exactly what it is we’re trying to stay away from.","support":"source-stated"}],"limitations":["Text as retrieved on October 23, 2026; originally published September 24, 2026.","Alexander describes possible safety risks and proposes possible taboos; the piece does not establish the scale of any resulting risk or whether companies will adopt them."],"mediaId":null,"correctionReason":null,"id":"607e571864c3b6803e040ee04630e117861dde6d2c1da761a53147d8bb7ab085","publicationId":"853460ae-2ef8-310c-fa8c-74cb826ed931","version":1,"sourceUrl":"https://www.astralcodexten.com/p/the-specter-of-neuralese","sourceTitle":"The Specter Of Neuralese","publisher":"Scott Alexander","publishedAt":"2026-09-24T22:12:39.000Z","originalPublished":{"value":"2026-09-24T22:12:39.000Z","precision":"instant","basis":"backfill:feed_published"},"firstObservedAt":"2026-10-23T23:05:57.734Z","createdAt":"2026-11-05T14:33:24.563Z","historical":true,"entityIds":["scott-alexander"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.astralcodexten.com/p/the-specter-of-neuralese","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"fec56d1650120a9d5125c46be40fdeeac91504549f007dca0b88ddf07e93a12c","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-23T23:05:55.108Z"},"provenance":{"textHash":"fafc68b668078328adaed0d28e255f141fb6e040802e81da6d732b7083349920","observationId":"9a216f7bb70de3350d2b446a9478d92a17225c9a53553dc430111ab11f92dfe2","retrievedAt":"2026-10-23T23:07:45.376Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"fd97aaaf-ecac-0390-3f07-ac4d2af7aba8"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"4459ad41-e416-9088-be3c-42148dbc4131"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-05T14:28:26.533Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"853460ae-2ef8-310c-fa8c-74cb826ed931","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"This is a single essay by Scott Alexander on Astral Codex Ten. The reading attributes its arguments and proposals to him and does not present them as established fact.","excerpt":"Linch says: there’s an emerging taboo on creating the dangerous sort of neuralese AIs seen in AI 2027."},{"dimension":"summary","status":"uncertain","explanation":"Astra is never identified as OpenAI's model, so readers must infer what it is. The reading also omits the main argument the essay responds to: OpenAI chief scientist Jakub Pachocki's defense that Astra's depth is within a factor of two of GPT-4, so its recurrence is no worse than adding normal layers. Alexander concedes this may not matter in real life but argues it matters for maintaining a taboo. Smallest repair: name Astra as OpenAI's looped model and add one sentence stating Pachocki's position and Alexander's taboo-based reply.","excerpt":"Pachocki says that Astra’s recurrence barely gives it any more layers than the competition."},{"dimension":"date","status":"pass","explanation":"The originalPublished instant, 2026-09-24T22:12:39Z, falls on the UTC calendar day of September 24, 2026, matching the source dateline. The reading uses no later events and no hindsight framing. The voluntary-commitment idea is presented as Alexander's suggestion, not an outcome. modifiedAt equals the publication time.","excerpt":"Sep 24, 2026"},{"dimension":"support","status":"pass","explanation":"The following claims are grounded in the text: the distinction between looped transformers and true neuralese, the claim that loops allow plotting between chain-of-thought steps, the smaller gain from simulated layers than from real ones, the three proposed taboos, and the voluntary-commitment suggestion. The stated unknowns are appropriately hedged.","excerpt":"An AI that uses loops to simulate a large number of layers (“looped transformer”) can make a plot within a single forward pass, in between chain-of-thought steps."},{"dimension":"media","status":"pass","explanation":"No media was selected and no verified candidates were supplied, so there is nothing to check.","excerpt":null}],"createdAt":"2026-11-05T14:28:26.533Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}