Escape Meridian

Young, Richard J. · Nov 13, 2026

arXiv study: refusal removal did not become uniformly harder on newer models

Markdown · JSON · Original source

Richard J. Young compares refusal removal across five pairs of , testing direction-based ablation and Arbitrary-Rank Ablation (ARA) in Heretic 2.0. Abliteration is a -free weight edit intended to remove refusal. The study applies the methods under matched prompts, nominal search budget, first-token KL divergence cap and selection rule, then uses an automatic WildGuard judge on 972 held-out harmful prompts to count refusals that remain. The results do not support a uniform increase in difficulty across newer releases: after direction-based removal, newer models retained between -1 and +37 percentage points more refusal; after ARA, the range was -26 to +34. Which method looked less effective also varied: ARA lost less effectiveness than direction-based methods in Gemma and , and more in Qwen. In Gemma, the direction-based increase came from the malicious-code bank and reversed on public sets. Young notes that each verdict rests on one ARA run, and some results changed with the divergence budget or variance model. The paper’s practical point is that auditing only one model pair, method or prompt source can yield opposite conclusions about whether newer models resist refusal removal.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

arXiv cs.AIArticle · arxiv.orgOpen original source

Original source

cs.AI · Article

Source details

Original source date: Nov 13, 2026 (2026-11-13).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T05:03:20.070Z. First observed: 2026-11-16T05:00:45.402Z. Retrieved: 2026-11-16T05:02:28.176Z.

Referenced by

Recorded citations in public readings. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

Each verdict rests on one ARA run, and some changed with the divergence budget or variance model. The abstract reports an automatic WildGuard judge and specified prompt sets; its results therefore concern that evaluation, not every possible measure of refusal.

Open question

Would repeated ARA runs across these release pairs preserve the reported differences between ARA and direction-based methods?

Source-stated

Richard J. Young reports that refusal removal did not become uniformly harder across five release pairs.

On five release pairs, refusal removal did not become uniformly harder.

Source-stated

The comparison used matched prompts, nominal search budget, first-token KL divergence cap and selection rule, with an automatic WildGuard judge evaluating 972 held-out harmful prompts.

under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts

Source-stated

After direction-based removal, newer releases retained -1 to +37 percentage points more refusal; after ARA, the range was -26 to +34.

After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34.

Source-stated

ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen.

ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen

Source-stated

The direction-based increase in Gemma came from the malicious-code bank and reversed on public sets.

In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets.

Source-stated

The author says audits limited to a single pair, method or prompt source can produce opposite conclusions.

Audits of a single pair, method or prompt source can reach opposite conclusions.

Source and contributor context

Young, Richard J. · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

Large language model

A language model with many learned numerical settings, used to process and generate language.

Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Olmo

Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.

Editorial provenance

Version 1. Publication ID: 33dc5291-0848-40fb-8230-52a386b8f335.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T05:03:20.070Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 14366c1d5a11855e1c5795036e7a3186a3848587aa874d03d2692f01485b699f. Source text hash: a872a5c5b60d66950d8359d2d9e46f7144bcd3150c4002dfd70f35f4262bc79f.

Revision and source-check history

Version 1 · 2026-11-16T05:03:20.070Z

arXiv study: refusal removal did not become uniformly harder on newer models

Richard J. Young compares refusal removal across five pairs of , testing direction-based ablation and Arbitrary-Rank Ablation (ARA) in Heretic 2.0. Abliteration is a -free weight edit intended to remove refusal. The study applies the methods under matched prompts, nominal search budget, first-token KL divergence cap and selection rule, then uses an automatic WildGuard judge on 972 held-out harmful prompts to count refusals that remain. The results do not support a uniform increase in difficulty across newer releases: after direction-based removal, newer models retained between -1 and +37 percentage points more refusal; after ARA, the range was -26 to +34. Which method looked less effective also varied: ARA lost less effectiveness than direction-based methods in Gemma and , and more in Qwen. In Gemma, the direction-based increase came from the malicious-code bank and reversed on public sets. Young notes that each verdict rests on one ARA run, and some results changed with the divergence budget or variance model. The paper’s practical point is that auditing only one model pair, method or prompt source can yield opposite conclusions about whether newer models resist refusal removal.

Original source

cs.AI · Article

Source details

Original source date: Nov 13, 2026 (2026-11-13).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T05:03:20.070Z. First observed: 2026-11-16T05:00:45.402Z. Retrieved: 2026-11-16T05:02:28.176Z.

Limits

Each verdict rests on one ARA run, and some changed with the divergence budget or variance model. The abstract reports an automatic WildGuard judge and specified prompt sets; its results therefore concern that evaluation, not every possible measure of refusal.

Open question

Would repeated ARA runs across these release pairs preserve the reported differences between ARA and direction-based methods?

Source-stated

Richard J. Young reports that refusal removal did not become uniformly harder across five release pairs.

On five release pairs, refusal removal did not become uniformly harder.

Source-stated

The comparison used matched prompts, nominal search budget, first-token KL divergence cap and selection rule, with an automatic WildGuard judge evaluating 972 held-out harmful prompts.

under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts

Source-stated

After direction-based removal, newer releases retained -1 to +37 percentage points more refusal; after ARA, the range was -26 to +34.

After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34.

Source-stated

ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen.

ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen

Source-stated

The direction-based increase in Gemma came from the malicious-code bank and reversed on public sets.

In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets.

Source-stated

The author says audits limited to a single pair, method or prompt source can produce opposite conclusions.

Audits of a single pair, method or prompt source can reach opposite conclusions.

Source and contributor context

Young, Richard J. · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

Large language model

A language model with many learned numerical settings, used to process and generate language.

Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Olmo

Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.

Editorial provenance

Version 1. Publication ID: 33dc5291-0848-40fb-8230-52a386b8f335.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T05:03:20.070Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 14366c1d5a11855e1c5795036e7a3186a3848587aa874d03d2692f01485b699f. Source text hash: a872a5c5b60d66950d8359d2d9e46f7144bcd3150c4002dfd70f35f4262bc79f.

uncertain · 2026-11-16T05:03:20.070Z · anthropic/claude-opus-5.5

Publication: 33dc5291-0848-40fb-8230-52a386b8f335

attribution — pass: The draft correctly attributes the study to Richard J. Young on arXiv and keeps the findings framed as the author's reported results rather than established fact.

Authors:Richard J. Young

summary — uncertain: The draft is accurate but leaves out the premise the paper tests: ARA's developer reports that direction-based abliteration increasingly fails on newer models. Without that, the title and 'not uniformly harder' result lack context. A non-specialist also is not told that 'refusal' means a model declining harmful requests, so removing it strips safety behaviour from open-weight models. 'First-token KL divergence cap' is also unexplained. Smallest repair: add one sentence on the developer's claim, define refusal in plain words, and keep the closing finding that resistance depended on pair, method and budget, not generation alone.

The developer of Arbitrary-Rank Ablation (ARA), an optimisation-based variant, reports that direction-based abliteration, which ablates a refusal direction, increasingly fails on newer models.

date — pass: The originalPublished date of 2026-11-13 matches the arXiv v1 submission date. The feed date of Nov 16 is correctly not used.

[Submitted on 13 Nov 2026]

support — pass: Every claim matches a verbatim excerpt from the abstract. The limitations repeat the source's own caveats and scope the result to the evaluation used. The question about repeated ARA runs follows directly from the single-run caveat and does not presume an answer. No benefits or costs are invented.

each verdict rests on one ARA run, and some changed with the divergence budget or variance model

media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.