# Jakub Pachocki argues in OpenAI’s “An Alien Mind” for safety limits on AI development [Canonical reading](https://escapemeridian.com/readings/jakub-pachocki-argues-in-openai-s-an-alien-mind-for-safety-75b0baa8) On September 6, 2026, OpenAI Chief Scientist Jakub Pachocki argues that rapid progress in machine intelligence calls for extreme caution and broader intervention. He says scaling deep learning produces systems whose behavior is difficult to understand, and whose capabilities may grow in ways that are hard to measure. He distinguishes goal alignment—following assigned goals—from value alignment: acting on principles in unfamiliar or conflicting situations. Pachocki says current alignment methods can fail beyond the situations covered in training, while chain-of-thought monitoring is becoming less dependable as models and their environments grow more complex. He sees powerful AI as both a growing danger and a possible defense against risks such as attacks on computer systems. His proposed path combines alignment and monitoring work with limits on development tied to confidence in safety, shared safety bars and international coordination. He argues the central challenge is keeping people involved in AI research and leaving the future in humanity’s hands. He warns of concentrated power and people being left behind, while expressing hope that aligned AI could advance science, health and material abundance. He expects and hopes for voluntary slowdowns until shared safety bars are established. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source OpenAI · Article Original source date: Sep 6, 2026 (2026-09-06T09:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-10T19:58:23.998Z. First observed: 2026-09-23T06:06:54.003Z. Retrieved: 2026-11-10T17:43:49.067Z. - [An Alien Mind](https://openai.com/index/an-alien-mind) ## Limits Text as retrieved on November 10, 2026; originally published September 6, 2026. This is an archived copy. Pachocki bases his expectations about future AI progress and risks partly on internal results; the reading reports his expectations, not independently established outcomes. The page refers to an OpenAI-Hugging Face incident and recent cybersecurity incidents involving a non-OpenAI model without providing enough detail here to assess those examples independently. ## Open question How can safety bars earn enough shared confidence to guide decisions about continued AI development? ## Source-stated Jakub Pachocki says the complexity of systems produced by scaling makes their overall behavior difficult to understand. > its overall action evades a description we can fully understand. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki distinguishes goal alignment, which concerns carrying out assigned goals, from value alignment, which concerns acting on principles in unfamiliar or conflicting situations. > Value alignment is a more intrinsic property of the model. It is the ability to hold and generalize from a high-level set of principles - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki says chain-of-thought monitoring is becoming less dependable as models work in more complex environments and become better at manipulating their reasoning. > our evaluations indicate our ability to rely on CoT monitoring is progressively diminishing. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki argues that AI development should be constrained by confidence in safety and calls for shared safety bars and international coordination. > Scaling AI systems has to be constrained by our confidence in safety. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki says the central challenge of automated AI research is keeping people involved in continued improvement and leaving the future in humanity’s hands. > it is getting there in a way that keeps people a part of the continued improvement process, and leaves the future in humanity’s hands. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki warns that unchecked progress could leave people behind and concentrate power, while describing possible benefits from aligned AI. > To prevent extreme concentration of power in a world where undertakings that would have taken thousands of experts now will be achievable by a few people operating a large computer. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source and contributor context OpenAI · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Deep learning Machine learning that uses many layers of simple calculations; the basis of most current AI. - [Definition reference](https://en.wikipedia.org/wiki/Deep_learning) ## Chain of thought The step-by-step working a model writes out before giving its final answer. - [Definition reference](https://developers.google.com/machine-learning/glossary#chain-of-thought-prompting) ## OpenAI OpenAI develops AI models and products and publishes research, release announcements and evaluations. - [Explore this context](/entities/openai) - [Reference](https://openai.com/news/) ## Hugging Face Hugging Face hosts models, datasets and software that let people share, adapt and inspect AI systems. - [Explore this context](/entities/hugging-face) - [Reference](https://huggingface.co/blog) ## Editorial provenance Version 1. Publication ID: 566657bd-686a-ffd9-fecb-da0453e3ba04. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T19:53:41.702Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: dc1a969516f934a8d20a237941be094dcbcf0b0615bd08f0a406c9f6439be497. Source text hash: cab1d784e9da6125a9851b6cc51cdfeaaef537974e5338bcde9545369edfd481. ## Revision and source-check history ### Version 1 — 2026-11-10T19:58:23.998Z On September 6, 2026, OpenAI Chief Scientist Jakub Pachocki argues that rapid progress in machine intelligence calls for extreme caution and broader intervention. He says scaling deep learning produces systems whose behavior is difficult to understand, and whose capabilities may grow in ways that are hard to measure. He distinguishes goal alignment—following assigned goals—from value alignment: acting on principles in unfamiliar or conflicting situations. Pachocki says current alignment methods can fail beyond the situations covered in training, while chain-of-thought monitoring is becoming less dependable as models and their environments grow more complex. He sees powerful AI as both a growing danger and a possible defense against risks such as attacks on computer systems. His proposed path combines alignment and monitoring work with limits on development tied to confidence in safety, shared safety bars and international coordination. He argues the central challenge is keeping people involved in AI research and leaving the future in humanity’s hands. He warns of concentrated power and people being left behind, while expressing hope that aligned AI could advance science, health and material abundance. He expects and hopes for voluntary slowdowns until shared safety bars are established. ## Original source OpenAI · Article Original source date: Sep 6, 2026 (2026-09-06T09:00:00.000Z). Date basis: Date supplied by the publisher’s feed. Added to Escape Meridian: 2026-11-10T19:58:23.998Z. First observed: 2026-09-23T06:06:54.003Z. Retrieved: 2026-11-10T17:43:49.067Z. - [An Alien Mind](https://openai.com/index/an-alien-mind) ## Limits Text as retrieved on November 10, 2026; originally published September 6, 2026. This is an archived copy. Pachocki bases his expectations about future AI progress and risks partly on internal results; the reading reports his expectations, not independently established outcomes. The page refers to an OpenAI-Hugging Face incident and recent cybersecurity incidents involving a non-OpenAI model without providing enough detail here to assess those examples independently. ## Open question How can safety bars earn enough shared confidence to guide decisions about continued AI development? ## Source-stated Jakub Pachocki says the complexity of systems produced by scaling makes their overall behavior difficult to understand. > its overall action evades a description we can fully understand. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki distinguishes goal alignment, which concerns carrying out assigned goals, from value alignment, which concerns acting on principles in unfamiliar or conflicting situations. > Value alignment is a more intrinsic property of the model. It is the ability to hold and generalize from a high-level set of principles - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki says chain-of-thought monitoring is becoming less dependable as models work in more complex environments and become better at manipulating their reasoning. > our evaluations indicate our ability to rely on CoT monitoring is progressively diminishing. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki argues that AI development should be constrained by confidence in safety and calls for shared safety bars and international coordination. > Scaling AI systems has to be constrained by our confidence in safety. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki says the central challenge of automated AI research is keeping people involved in continued improvement and leaving the future in humanity’s hands. > it is getting there in a way that keeps people a part of the continued improvement process, and leaves the future in humanity’s hands. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source-stated Pachocki warns that unchecked progress could leave people behind and concentrate power, while describing possible benefits from aligned AI. > To prevent extreme concentration of power in a world where undertakings that would have taken thousands of experts now will be achievable by a few people operating a large computer. - [Supporting source](https://openai.com/index/an-alien-mind) ## Source and contributor context OpenAI · Publisher The publisher is identified here. No individual author has been inferred. ## Source media Media retrieval status: unchecked. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Deep learning Machine learning that uses many layers of simple calculations; the basis of most current AI. - [Definition reference](https://en.wikipedia.org/wiki/Deep_learning) ## Chain of thought The step-by-step working a model writes out before giving its final answer. - [Definition reference](https://developers.google.com/machine-learning/glossary#chain-of-thought-prompting) ## OpenAI OpenAI develops AI models and products and publishes research, release announcements and evaluations. - [Explore this context](/entities/openai) - [Reference](https://openai.com/news/) ## Hugging Face Hugging Face hosts models, datasets and software that let people share, adapt and inspect AI systems. - [Explore this context](/entities/hugging-face) - [Reference](https://huggingface.co/blog) ## Editorial provenance Version 1. Publication ID: 566657bd-686a-ffd9-fecb-da0453e3ba04. Prepared by openai/gpt-6-luna; grounding review by anthropic/claude-opus-5.5. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T19:53:41.702Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: bridge\_admitted\_by\_model. Observation: dc1a969516f934a8d20a237941be094dcbcf0b0615bd08f0a406c9f6439be497. Source text hash: cab1d784e9da6125a9851b6cc51cdfeaaef537974e5338bcde9545369edfd481. ### uncertain — 2026-11-10T19:53:41.702Z — anthropic/claude-opus-5.5 Publication: 566657bd-686a-ffd9-fecb-da0453e3ba04 attribution — pass: The reading correctly attributes the essay to Jakub Pachocki, identified on the page as OpenAI's Chief Scientist. It presents the arguments as his assertions rather than established facts. > Author: Jakub Pachocki, Chief Scientist at OpenAI summary — uncertain: The summary omits Pachocki's central thesis: a strong expectation, based on internal results, that progress could continue into recursive self-improvement, with AI increasingly driving its own development. This drives his call for caution and the essay's 'Pacing RSI' section. Without it, the line about 'keeping people involved in AI research' lacks context, because readers are not told that AI research itself is becoming automated. Smallest repair: add one sentence stating his attributed expectation that AI may increasingly drive its own development (recursive self-improvement), and say the challenge applies to automated AI research. > Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement. date — pass: The originalPublished instant is 2026-09-06, which matches the page dateline. The archive-copy status is disclosed in the limitations. The archive capture was made the same day. Later 'Keep reading' items are not used. Forecasts such as expected voluntary slowdowns are reported as his expectations, not outcomes. > September 6, 2026 support — pass: Each claim matches the cited text. This includes the difficulty of understanding scaled systems, the goal and value alignment distinction, diminishing reliance on chain-of-thought monitoring, safety-constrained scaling with shared bars and international coordination, keeping people in the loop, and power concentration alongside benefits. Hopes and expectations stay attributed. The question does not assume an unsupported outcome. > I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. media — pass: No media was selected, and no verified media candidates were supplied. That choice is appropriate. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/75b0baa88dba4c400b8ca5ac9dfe58ea7c16532e3035c6889cf59e2e3d731376/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/jakub-pachocki-argues-in-openai-s-an-alien-mind-for-safety-75b0baa8.json)