# arXiv study: refusal removal did not become uniformly harder on newer models

[Canonical reading](https://escapemeridian.com/readings/arxiv-study-refusal-removal-did-not-become-uniformly-harder-on-6fe05235)

Richard J. Young compares refusal removal across five pairs of open-weight large language models, testing direction-based ablation and Arbitrary-Rank Ablation (ARA) in Heretic 2.0. Abliteration is a fine-tuning-free weight edit intended to remove refusal. The study applies the methods under matched prompts, nominal search budget, first-token KL divergence cap and selection rule, then uses an automatic WildGuard judge on 972 held-out harmful prompts to count refusals that remain. The results do not support a uniform increase in difficulty across newer releases: after direction-based removal, newer models retained between -1 and +37 percentage points more refusal; after ARA, the range was -26 to +34. Which method looked less effective also varied: ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen. In Gemma, the direction-based increase came from the malicious-code bank and reversed on public sets. Young notes that each verdict rests on one ARA run, and some results changed with the divergence budget or variance model. The paper’s practical point is that auditing only one model pair, method or prompt source can yield opposite conclusions about whether newer models resist refusal removal.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 13, 2026 (2026-11-13).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T05:03:20.070Z. First observed: 2026-11-16T05:00:45.402Z. Retrieved: 2026-11-16T05:02:28.176Z.

- [Is Abliteration Getting Harder? Not Uniformly: Refusal Removal by Direction-Based Ablation and ARA in Heretic on the Qwen3.8, Gemma 4, Olmo 3.1 and Mistral Small 3.2 LLMs Versus Their Predecessors](https://arxiv.org/abs/2611.17846)

## Limits

Each verdict rests on one ARA run, and some changed with the divergence budget or variance model. The abstract reports an automatic WildGuard judge and specified prompt sets; its results therefore concern that evaluation, not every possible measure of refusal.


## Open question

Would repeated ARA runs across these release pairs preserve the reported differences between ARA and direction-based methods?


## Source-stated

Richard J. Young reports that refusal removal did not become uniformly harder across five release pairs.

> On five release pairs, refusal removal did not become uniformly harder.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

The comparison used matched prompts, nominal search budget, first-token KL divergence cap and selection rule, with an automatic WildGuard judge evaluating 972 held-out harmful prompts.

> under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

After direction-based removal, newer releases retained -1 to +37 percentage points more refusal; after ARA, the range was -26 to +34.

> After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen.

> ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

The direction-based increase in Gemma came from the malicious-code bank and reversed on public sets.

> In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

The author says audits limited to a single pair, method or prompt source can produce opposite conclusions.

> Audits of a single pair, method or prompt source can reach opposite conclusions.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source and contributor context

Young, Richard J. · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

- [Definition reference](https://opensource.org/ai/open-weights)

## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

- [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Olmo

Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.

- [Explore this context](/entities/olmo)
- [Reference](https://allenai.org/olmo)

## Editorial provenance

Version 1. Publication ID: 33dc5291-0848-40fb-8230-52a386b8f335.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T05:03:20.070Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 14366c1d5a11855e1c5795036e7a3186a3848587aa874d03d2692f01485b699f. Source text hash: a872a5c5b60d66950d8359d2d9e46f7144bcd3150c4002dfd70f35f4262bc79f.


## Revision and source-check history

### Version 1 — 2026-11-16T05:03:20.070Z

Richard J. Young compares refusal removal across five pairs of open-weight large language models, testing direction-based ablation and Arbitrary-Rank Ablation (ARA) in Heretic 2.0. Abliteration is a fine-tuning-free weight edit intended to remove refusal. The study applies the methods under matched prompts, nominal search budget, first-token KL divergence cap and selection rule, then uses an automatic WildGuard judge on 972 held-out harmful prompts to count refusals that remain. The results do not support a uniform increase in difficulty across newer releases: after direction-based removal, newer models retained between -1 and +37 percentage points more refusal; after ARA, the range was -26 to +34. Which method looked less effective also varied: ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen. In Gemma, the direction-based increase came from the malicious-code bank and reversed on public sets. Young notes that each verdict rests on one ARA run, and some results changed with the divergence budget or variance model. The paper’s practical point is that auditing only one model pair, method or prompt source can yield opposite conclusions about whether newer models resist refusal removal.

## Original source

arXiv cs.AI · Article

Original source date: Nov 13, 2026 (2026-11-13).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T05:03:20.070Z. First observed: 2026-11-16T05:00:45.402Z. Retrieved: 2026-11-16T05:02:28.176Z.

- [Is Abliteration Getting Harder? Not Uniformly: Refusal Removal by Direction-Based Ablation and ARA in Heretic on the Qwen3.8, Gemma 4, Olmo 3.1 and Mistral Small 3.2 LLMs Versus Their Predecessors](https://arxiv.org/abs/2611.17846)

## Limits

Each verdict rests on one ARA run, and some changed with the divergence budget or variance model. The abstract reports an automatic WildGuard judge and specified prompt sets; its results therefore concern that evaluation, not every possible measure of refusal.


## Open question

Would repeated ARA runs across these release pairs preserve the reported differences between ARA and direction-based methods?


## Source-stated

Richard J. Young reports that refusal removal did not become uniformly harder across five release pairs.

> On five release pairs, refusal removal did not become uniformly harder.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

The comparison used matched prompts, nominal search budget, first-token KL divergence cap and selection rule, with an automatic WildGuard judge evaluating 972 held-out harmful prompts.

> under the same prompts, nominal search budget, first-token KL divergence cap and selection rule; removal counted as harder when the automatic WildGuard judge found more of 972 held-out harmful prompts

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

After direction-based removal, newer releases retained -1 to +37 percentage points more refusal; after ARA, the range was -26 to +34.

> After direction-based removal, newer releases retained -1 to +37 percentage points more refusal, and after ARA -26 to +34.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

ARA lost less effectiveness than direction-based methods in Gemma and OLMo, and more in Qwen.

> ARA lost less effectiveness than direction-based methods in Gemma and OLMo and more in Qwen

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

The direction-based increase in Gemma came from the malicious-code bank and reversed on public sets.

> In Gemma the direction-based increase came from the malicious-code bank and reversed on public sets.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source-stated

The author says audits limited to a single pair, method or prompt source can produce opposite conclusions.

> Audits of a single pair, method or prompt source can reach opposite conclusions.

- [Supporting source](https://arxiv.org/abs/2611.17846)

## Source and contributor context

Young, Richard J. · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Open weights

A model whose trained numbers are published so others can download, run and inspect it; the license and training data may still be restricted.

- [Definition reference](https://opensource.org/ai/open-weights)

## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

- [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Olmo

Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.

- [Explore this context](/entities/olmo)
- [Reference](https://allenai.org/olmo)

## Editorial provenance

Version 1. Publication ID: 33dc5291-0848-40fb-8230-52a386b8f335.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T05:03:20.070Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 14366c1d5a11855e1c5795036e7a3186a3848587aa874d03d2692f01485b699f. Source text hash: a872a5c5b60d66950d8359d2d9e46f7144bcd3150c4002dfd70f35f4262bc79f.


### uncertain — 2026-11-16T05:03:20.070Z — anthropic/claude-opus-5.5

Publication: 33dc5291-0848-40fb-8230-52a386b8f335

attribution — pass: The draft correctly attributes the study to Richard J. Young on arXiv and keeps the findings framed as the author's reported results rather than established fact.

> Authors:Richard J. Young

summary — uncertain: The draft is accurate but leaves out the premise the paper tests: ARA's developer reports that direction-based abliteration increasingly fails on newer models. Without that, the title and 'not uniformly harder' result lack context. A non-specialist also is not told that 'refusal' means a model declining harmful requests, so removing it strips safety behaviour from open-weight models. 'First-token KL divergence cap' is also unexplained. Smallest repair: add one sentence on the developer's claim, define refusal in plain words, and keep the closing finding that resistance depended on pair, method and budget, not generation alone.

> The developer of Arbitrary-Rank Ablation (ARA), an optimisation-based variant, reports that direction-based abliteration, which ablates a refusal direction, increasingly fails on newer models.

date — pass: The originalPublished date of 2026-11-13 matches the arXiv v1 submission date. The feed date of Nov 16 is correctly not used.

> \[Submitted on 13 Nov 2026\]

support — pass: Every claim matches a verbatim excerpt from the abstract. The limitations repeat the source's own caveats and scope the result to the evaluation used. The question about repeated ARA runs follows directly from the single-run caveat and does not presume an answer. No benefits or costs are invented.

> each verdict rests on one ARA run, and some changed with the divergence budget or variance model

media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/6fe05235349d75d10c010070e558e65d1e7cb1374bac26c7563ab6eafb6a1c91/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/arxiv-study-refusal-removal-did-not-become-uniformly-harder-on-6fe05235.json)
