Redwood Research · Oct 16, 2026
Redwood Research’s Screwtape finds monitor blind spots by editing real agent transcripts
Markdown · JSON · Original source
Gosha Kozhevnikov and Tyler describe Screwtape, a pipeline that edits real AI-agent transcripts to create harmful actions that evade monitors—systems that inspect an agent’s actions but not its reasoning. An attacker repeatedly revises each edit using feedback from a monitor, while a separate language-model verifier checks whether the proposed attack is plausible and severe enough. In tests on 94 transcripts, the team says it found attacks that evaded its own monitors and a held-out monitor from a frontier developer; human reviewers confirmed some of the latter. The authors see the clearest use as finding specific blind spots developers can patch, not measuring overall safety: the method searches only a limited part of the possible attack space, and finding no attack provides little assurance. They also describe a tradeoff: stronger monitors may catch harmful actions, but could also help developers advance capable systems without addressing underlying misalignment. The post says the pipeline does not yet handle reasoning traces and that its final attacks still require human review.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
Redwood Research · Article
Source details
Original source date: Oct 16, 2026 (2026-10-16T16:01:26+00:00).
Date basis: Date stated by the publisher.
Added to Escape Meridian: 2026-10-19T13:02:24.396Z. First observed: 2026-10-18T02:34:54.300Z. Retrieved: 2026-10-19T13:01:28.795Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The post also says human review confirmed 6 of 17 attacks that evaded the held-out monitor, while its earlier summary says six of 35 attacks evaded that monitor and were confirmed. The relationship between these denominators is not explained in the supplied text. The pipeline lacks access to the original environment, relies on an LLM verifier that can accept invalid attacks, and requires human review of resulting attacks. Screwtape currently generates transcripts without reasoning traces, limiting its applicability to monitors that inspect both actions and reasoning. The authors caution that the pipeline’s findings do not quantify real-world failure rates; failure to find an attack provides little information about safety.
Open question
What additional methodology would be needed to turn attacks found by Screwtape into a credible estimate of how often monitors fail in deployment?
Source-stated
Screwtape edits real agent transcripts and iterates attacks against an action-only monitor, using a separate language-model verifier to assess whether edits are plausible and harmful.
a pipeline that turns agent transcripts into realistic attacks that evade action-only monitors
Source-stated
The authors report that tests on 94 transcripts produced 35 attacks that evaded their monitors; six also evaded a held-out monitor and were confirmed through human review.
This produced 35 attacks that evaded our monitors. Six of these 35 attacks also evaded the frontier developer’s monitor, and we confirmed them as valid through human review.
Source-stated
The authors say Screwtape is more useful for finding monitor weaknesses to patch than for estimating safety, because it cannot provide quantitative safety estimates and searches only a limited attack space.
In its current form, the pipeline can’t show how often a monitor fails in normal deployment or produce a numerical safety estimate.
Source-stated
The authors are uncertain whether helping developers strengthen frontier monitors is beneficial, since they could improve short-term safety while allowing capability development to continue without addressing underlying misalignment.
Strong monitors increase short-term safety by catching malign actions. On the other hand, they might let developers keep improving capabilities without addressing the underlying misalignment
Source and contributor context
Redwood Research · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media was declared by the source, but no asset passed the supported availability checks. Open the original source to view it.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Redwood Research
Specific empirical AI-control and monitor-failure work, including counterexamples.
Editorial provenance
Version 1. Publication ID: 1a85fff9-d6a9-404c-8ea3-70f797d944c6.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-10-19T13:02:24.396Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 731ec8394c162cbeb1068e6be6a500b6035a187988ac7fa9d1be7953052f4712. Source text hash: fdb3f4375abc89bb5e359b1266dc3bcb0c05031454ee1d2ffa14942c7fe1968a.
Revision and source-check history
Version 1 · 2026-10-19T13:02:24.396Z
Redwood Research’s Screwtape finds monitor blind spots by editing real agent transcripts
Gosha Kozhevnikov and Tyler describe Screwtape, a pipeline that edits real AI-agent transcripts to create harmful actions that evade monitors—systems that inspect an agent’s actions but not its reasoning. An attacker repeatedly revises each edit using feedback from a monitor, while a separate language-model verifier checks whether the proposed attack is plausible and severe enough. In tests on 94 transcripts, the team says it found attacks that evaded its own monitors and a held-out monitor from a frontier developer; human reviewers confirmed some of the latter. The authors see the clearest use as finding specific blind spots developers can patch, not measuring overall safety: the method searches only a limited part of the possible attack space, and finding no attack provides little assurance. They also describe a tradeoff: stronger monitors may catch harmful actions, but could also help developers advance capable systems without addressing underlying misalignment. The post says the pipeline does not yet handle reasoning traces and that its final attacks still require human review.
Original source
Redwood Research · Article
Source details
Original source date: Oct 16, 2026 (2026-10-16T16:01:26+00:00).
Date basis: Date stated by the publisher.
Added to Escape Meridian: 2026-10-19T13:02:24.396Z. First observed: 2026-10-18T02:34:54.300Z. Retrieved: 2026-10-19T13:01:28.795Z.
Limits
The post also says human review confirmed 6 of 17 attacks that evaded the held-out monitor, while its earlier summary says six of 35 attacks evaded that monitor and were confirmed. The relationship between these denominators is not explained in the supplied text. The pipeline lacks access to the original environment, relies on an LLM verifier that can accept invalid attacks, and requires human review of resulting attacks. Screwtape currently generates transcripts without reasoning traces, limiting its applicability to monitors that inspect both actions and reasoning. The authors caution that the pipeline’s findings do not quantify real-world failure rates; failure to find an attack provides little information about safety.
Open question
What additional methodology would be needed to turn attacks found by Screwtape into a credible estimate of how often monitors fail in deployment?
Source-stated
Screwtape edits real agent transcripts and iterates attacks against an action-only monitor, using a separate language-model verifier to assess whether edits are plausible and harmful.
a pipeline that turns agent transcripts into realistic attacks that evade action-only monitors
Source-stated
The authors report that tests on 94 transcripts produced 35 attacks that evaded their monitors; six also evaded a held-out monitor and were confirmed through human review.
This produced 35 attacks that evaded our monitors. Six of these 35 attacks also evaded the frontier developer’s monitor, and we confirmed them as valid through human review.
Source-stated
The authors say Screwtape is more useful for finding monitor weaknesses to patch than for estimating safety, because it cannot provide quantitative safety estimates and searches only a limited attack space.
In its current form, the pipeline can’t show how often a monitor fails in normal deployment or produce a numerical safety estimate.
Source-stated
The authors are uncertain whether helping developers strengthen frontier monitors is beneficial, since they could improve short-term safety while allowing capability development to continue without addressing underlying misalignment.
Strong monitors increase short-term safety by catching malign actions. On the other hand, they might let developers keep improving capabilities without addressing the underlying misalignment
Source and contributor context
Redwood Research · Publisher
The publisher is identified here. No individual author has been inferred.
Source media
Media was declared by the source, but no asset passed the supported availability checks. Open the original source to view it.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Redwood Research
Specific empirical AI-control and monitor-failure work, including counterexamples.
Editorial provenance
Version 1. Publication ID: 1a85fff9-d6a9-404c-8ea3-70f797d944c6.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-10-19T13:02:24.396Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 731ec8394c162cbeb1068e6be6a500b6035a187988ac7fa9d1be7953052f4712. Source text hash: fdb3f4375abc89bb5e359b1266dc3bcb0c05031454ee1d2ffa14942c7fe1968a.
passed · 2026-10-19T13:02:24.396Z · anthropic/claude-opus-5.5
Publication: 1a85fff9-d6a9-404c-8ea3-70f797d944c6
attribution — pass: The draft correctly attributes the post to Gosha Kozhevnikov and Tyler, published by Redwood Research. It frames the results and judgments as the authors' reports and views.
Gosha Kozhevnikov and Tyler Oct 16, 2026
summary — pass: The summary accurately describes the pipeline and its components: the attacker, the verifier and the monitor. It explains action-only monitors in plain language. It states the results and the authors' main lessons, uncertainties and limitations. It also flags the 6-of-17 versus 6-of-35 denominator ambiguity. The title and question are grounded in the source, which says that quantifying safety information requires additional methodology.
quantifying it requires additional methodology that carefully accounts for approximations and uncertainties
date — pass: The originalPublished value of 2026-10-16 matches the post's byline date and the feed date. No earlier submission is indicated.
Gosha Kozhevnikov and Tyler Oct 16, 2026
support — pass: Each claim is directly supported by the source text. This covers the 94 transcripts, the 35 evading attacks and the 6 confirmed attacks. It also covers the inability to produce numerical safety estimates, the authors' uncertainty about strengthening monitors, the lack of reasoning-trace support and the need for human review. No benefits or costs are implied beyond what the source states.
We ran the pipeline 252 times on 94 unique transcripts. This produced 35 attacks that evaded our monitors.
media — pass: The selected media is the verified link to the original Redwood Research post. It is the direct and most relevant source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.