Young, Richard J. · Nov 13, 2026
arXiv study: refusal removal did not become uniformly harder on newer models
Markdown · JSON · Original source
Richard J. Young compares refusal removal across five pairs of , testing direction-based ablation and Arbitrary-Rank Ablation (ARA) in Heretic 2.0. Abliteration is a -free weight edit intended to remove refusal. The study applies the methods under matched prompts, nominal search budget, first-token KL divergence cap and selection rule, then uses an automatic WildGuard judge on 972 held-out harmful prompts to count refusals that remain. The results do not support a uniform increase in difficulty across newer releases: after direction-based removal, newer models retained between -1 and +37 percentage points more refusal; after ARA, the range was -26 to +34. Which method looked less effective also varied: ARA lost less effectiveness than direction-based methods in Gemma and , and more in Qwen. In Gemma, the direction-based increase came from the malicious-code bank and reversed on public sets. Young notes that each verdict rests on one ARA run, and some results changed with the divergence budget or variance model. The paper’s practical point is that auditing only one model pair, method or prompt source can yield opposite conclusions about whether newer models resist refusal removal.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 13, 2026 (2026-11-13).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-16T05:03:20.070Z. First observed: 2026-11-16T05:00:45.402Z. Retrieved: 2026-11-16T05:02:28.176Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
Each verdict rests on one ARA run, and some changed with the divergence budget or variance model. The abstract reports an automatic WildGuard judge and specified prompt sets; its results therefore concern that evaluation, not every possible measure of refusal.
Open question
Would repeated ARA runs across these release pairs preserve the reported differences between ARA and direction-based methods?
Source-stated
Richard J. Young reports that refusal removal did not become uniformly harder across five release pairs.
On five release pairs, refusal removal did not become uniformly harder.
Source-stated
The comparison used matched prompts, nominal search budget, first-token KL divergence cap and selection rule, with an automatic WildGuard judge evaluating 972 held-out harmful prompts.
under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts
Source-stated
After direction-based removal, newer releases retained -1 to +37 percentage points more refusal; after ARA, the range was -26 to +34.
After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34.
Source-stated
ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen.
ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen
Source-stated
The direction-based increase in Gemma came from the malicious-code bank and reversed on public sets.
In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets.
Source-stated
The author says audits limited to a single pair, method or prompt source can produce opposite conclusions.
Audits of a single pair, method or prompt source can reach opposite conclusions.
Source and contributor context
Young, Richard J. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Open weights
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Olmo
Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Editorial provenance
Version 1. Publication ID: 33dc5291-0848-40fb-8230-52a386b8f335.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T05:03:20.070Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 14366c1d5a11855e1c5795036e7a3186a3848587aa874d03d2692f01485b699f. Source text hash: a872a5c5b60d66950d8359d2d9e46f7144bcd3150c4002dfd70f35f4262bc79f.
Revision and source-check history
Version 1 · 2026-11-16T05:03:20.070Z
arXiv study: refusal removal did not become uniformly harder on newer models
Richard J. Young compares refusal removal across five pairs of , testing direction-based ablation and Arbitrary-Rank Ablation (ARA) in Heretic 2.0. Abliteration is a -free weight edit intended to remove refusal. The study applies the methods under matched prompts, nominal search budget, first-token KL divergence cap and selection rule, then uses an automatic WildGuard judge on 972 held-out harmful prompts to count refusals that remain. The results do not support a uniform increase in difficulty across newer releases: after direction-based removal, newer models retained between -1 and +37 percentage points more refusal; after ARA, the range was -26 to +34. Which method looked less effective also varied: ARA lost less effectiveness than direction-based methods in Gemma and , and more in Qwen. In Gemma, the direction-based increase came from the malicious-code bank and reversed on public sets. Young notes that each verdict rests on one ARA run, and some results changed with the divergence budget or variance model. The paper’s practical point is that auditing only one model pair, method or prompt source can yield opposite conclusions about whether newer models resist refusal removal.
Original source
cs.AI · Article
Source details
Original source date: Nov 13, 2026 (2026-11-13).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-16T05:03:20.070Z. First observed: 2026-11-16T05:00:45.402Z. Retrieved: 2026-11-16T05:02:28.176Z.
Limits
Each verdict rests on one ARA run, and some changed with the divergence budget or variance model. The abstract reports an automatic WildGuard judge and specified prompt sets; its results therefore concern that evaluation, not every possible measure of refusal.
Open question
Would repeated ARA runs across these release pairs preserve the reported differences between ARA and direction-based methods?
Source-stated
Richard J. Young reports that refusal removal did not become uniformly harder across five release pairs.
On five release pairs, refusal removal did not become uniformly harder.
Source-stated
The comparison used matched prompts, nominal search budget, first-token KL divergence cap and selection rule, with an automatic WildGuard judge evaluating 972 held-out harmful prompts.
under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts
Source-stated
After direction-based removal, newer releases retained -1 to +37 percentage points more refusal; after ARA, the range was -26 to +34.
After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34.
Source-stated
ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen.
ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen
Source-stated
The direction-based increase in Gemma came from the malicious-code bank and reversed on public sets.
In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets.
Source-stated
The author says audits limited to a single pair, method or prompt source can produce opposite conclusions.
Audits of a single pair, method or prompt source can reach opposite conclusions.
Source and contributor context
Young, Richard J. · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Open weights
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Olmo
Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Editorial provenance
Version 1. Publication ID: 33dc5291-0848-40fb-8230-52a386b8f335.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T05:03:20.070Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 14366c1d5a11855e1c5795036e7a3186a3848587aa874d03d2692f01485b699f. Source text hash: a872a5c5b60d66950d8359d2d9e46f7144bcd3150c4002dfd70f35f4262bc79f.
uncertain · 2026-11-16T05:03:20.070Z · anthropic/claude-opus-5.5
Publication: 33dc5291-0848-40fb-8230-52a386b8f335
attribution — pass: The draft correctly attributes the study to Richard J. Young on arXiv and keeps the findings framed as the author's reported results rather than established fact.
Authors:Richard J. Young
summary — uncertain: The draft is accurate but leaves out the premise the paper tests: ARA's developer reports that direction-based abliteration increasingly fails on newer models. Without that, the title and 'not uniformly harder' result lack context. A non-specialist also is not told that 'refusal' means a model declining harmful requests, so removing it strips safety behaviour from open-weight models. 'First-token KL divergence cap' is also unexplained. Smallest repair: add one sentence on the developer's claim, define refusal in plain words, and keep the closing finding that resistance depended on pair, method and budget, not generation alone.
The developer of Arbitrary-Rank Ablation (ARA), an optimisation-based variant, reports that direction-based abliteration, which ablates a refusal direction, increasingly fails on newer models.
date — pass: The originalPublished date of 2026-11-13 matches the arXiv v1 submission date. The feed date of Nov 16 is correctly not used.
[Submitted on 13 Nov 2026]
support — pass: Every claim matches a verbatim excerpt from the abstract. The limitations repeat the source's own caveats and scope the result to the evaluation used. The question about repeated ARA runs follows directly from the single-run caveat and does not presume an answer. No benefits or costs are invented.
each verdict rests on one ARA run, and some changed with the divergence budget or variance model
media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referenceOlmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Explore OlmoarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivA model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.
Definition referenceA language model with many learned numerical settings, used to process and generate language.
Definition referenceFurther training of an existing model to adapt it to a particular task or use.
Definition referenceOlmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Explore OlmoarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv