# Blind Spots Travel, Knowledge Stays Home: What RL Inherits from an LLM Judge

[Canonical reading](https://escapemeridian.com/readings/blind-spots-travel-knowledge-stays-home-what-rl-inherits-from-3f4d12d5)

Liang Yue and Eric A. Vance report that in 814 RL runs teaching 7-8B models to answer closed-book questions or abstain, an LLM judge’s influence depended on what its verdicts expressed. A rewarded wrong answer could be memorised for that question, but the judge’s knowledge of individual answers did not improve the model’s choices on new questions over a baseline that sampled 32 answers and abstained when they disagreed. By contrast, a judge’s pattern of rejecting correct answers about a category could travel: the authors say rejecting eleven of twelve correct training answers about capitals of former states led the model to refuse 98% of such new questions. They report that changing those twelve verdicts by hand shifted refusal from 5% to 90%, even when the judge accepted as many answers as the answer key. The model still answered 94% correctly when abstention was blocked, which the authors take to indicate the capitals were not forgotten. Their proposed alternative needs neither labels nor a judge: reward answers that most of the model’s own untrained samples agree on. The abstract says this earns the answer key’s reward and keeps the category, highlighting a distinction between transferring answer knowledge and transferring a tendency to refuse a kind of question.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 18, 2026 (2026-11-18).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-20T05:16:34.052Z. First observed: 2026-11-20T05:01:34.695Z. Retrieved: 2026-11-20T05:02:30.364Z.

- [Blind Spots Travel, Knowledge Stays Home: What RL Inherits from an LLM Judge](https://arxiv.org/abs/2611.25191)

## Limits

The supplied evidence is the paper’s abstract, not its full methods or results; it does not provide enough detail here to assess the experimental setup or the breadth of the findings.


## Open question

How broadly do the reported results hold beyond 7-8B models trained on closed-book questions or abstention?


## Source-stated

The authors report 814 RL runs teaching 7-8B models to answer closed-book questions or abstain.

> In 814 RL runs that teach 7-8B models to answer closed-book questions or abstain, we ask what passes.

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source-stated

The authors say knowledge about single answers did not improve choices on new questions over sampling 32 answers and abstaining when they disagreed.

> A model trained with the answer key, or with a far stronger judge, chooses which new questions to answer no better than the untrained model does by sampling 32 answers and abstaining where they disagree

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source-stated

They report that rejecting eleven of twelve correct training answers about former-state capitals led to refusals on 98% of new questions of that kind.

> A judge that rejects eleven of the twelve correct training answers about capitals of former states teaches the model to refuse 98% of such capitals on new questions

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source-stated

The authors propose rewarding answers that most of the model’s own untrained samples agree on, without labels or a judge.

> rewarding the model for giving the answer that most of its own untrained samples agree on, with no labels and no judge, earns the answer key's reward and keeps the category.

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source and contributor context

Yue, Liang, Vance, Eric A. · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Reinforcement learning

Training by trial and error, in which a system is rewarded for the outcomes its designers want.

- [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl)

## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: bcd15748-cfad-4ce2-8f72-b3486442212a.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-20T05:16:34.052Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 3f673eef44f9c5320e7a30fc0a252c71868d79ded43cffe0a17ef07bcdb13e0e. Source text hash: dbb0938c6e1ebd63d328a443c4c8a014a138fc5c2103aba3662d81123364ecf2.


## Revision and source-check history

### Version 1 — 2026-11-20T05:16:34.052Z

Liang Yue and Eric A. Vance report that in 814 RL runs teaching 7-8B models to answer closed-book questions or abstain, an LLM judge’s influence depended on what its verdicts expressed. A rewarded wrong answer could be memorised for that question, but the judge’s knowledge of individual answers did not improve the model’s choices on new questions over a baseline that sampled 32 answers and abstained when they disagreed. By contrast, a judge’s pattern of rejecting correct answers about a category could travel: the authors say rejecting eleven of twelve correct training answers about capitals of former states led the model to refuse 98% of such new questions. They report that changing those twelve verdicts by hand shifted refusal from 5% to 90%, even when the judge accepted as many answers as the answer key. The model still answered 94% correctly when abstention was blocked, which the authors take to indicate the capitals were not forgotten. Their proposed alternative needs neither labels nor a judge: reward answers that most of the model’s own untrained samples agree on. The abstract says this earns the answer key’s reward and keeps the category, highlighting a distinction between transferring answer knowledge and transferring a tendency to refuse a kind of question.

## Original source

arXiv cs.AI · Article

Original source date: Nov 18, 2026 (2026-11-18).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-20T05:16:34.052Z. First observed: 2026-11-20T05:01:34.695Z. Retrieved: 2026-11-20T05:02:30.364Z.

- [Blind Spots Travel, Knowledge Stays Home: What RL Inherits from an LLM Judge](https://arxiv.org/abs/2611.25191)

## Limits

The supplied evidence is the paper’s abstract, not its full methods or results; it does not provide enough detail here to assess the experimental setup or the breadth of the findings.


## Open question

How broadly do the reported results hold beyond 7-8B models trained on closed-book questions or abstention?


## Source-stated

The authors report 814 RL runs teaching 7-8B models to answer closed-book questions or abstain.

> In 814 RL runs that teach 7-8B models to answer closed-book questions or abstain, we ask what passes.

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source-stated

The authors say knowledge about single answers did not improve choices on new questions over sampling 32 answers and abstaining when they disagreed.

> A model trained with the answer key, or with a far stronger judge, chooses which new questions to answer no better than the untrained model does by sampling 32 answers and abstaining where they disagree

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source-stated

They report that rejecting eleven of twelve correct training answers about former-state capitals led to refusals on 98% of new questions of that kind.

> A judge that rejects eleven of the twelve correct training answers about capitals of former states teaches the model to refuse 98% of such capitals on new questions

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source-stated

The authors propose rewarding answers that most of the model’s own untrained samples agree on, without labels or a judge.

> rewarding the model for giving the answer that most of its own untrained samples agree on, with no labels and no judge, earns the answer key's reward and keeps the category.

- [Supporting source](https://arxiv.org/abs/2611.25191)

## Source and contributor context

Yue, Liang, Vance, Eric A. · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Reinforcement learning

Training by trial and error, in which a system is rewarded for the outcomes its designers want.

- [Definition reference](https://developers.google.com/machine-learning/glossary#reinforcement-learning-rl)

## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: bcd15748-cfad-4ce2-8f72-b3486442212a.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-20T05:16:34.052Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 3f673eef44f9c5320e7a30fc0a252c71868d79ded43cffe0a17ef07bcdb13e0e. Source text hash: dbb0938c6e1ebd63d328a443c4c8a014a138fc5c2103aba3662d81123364ecf2.


### uncertain — 2026-11-20T05:16:34.052Z — anthropic/claude-opus-5.5

Publication: bcd15748-cfad-4ce2-8f72-b3486442212a

attribution — pass: The draft correctly attributes the work to the paper's listed authors, Liang Yue and Eric A. Vance, and keeps the findings framed as their report ('report', 'the authors say', 'the abstract says').

> Authors:Liang Yue, Eric A. Vance

summary — uncertain: Central terms are not explained for a general reader: 'RL', '7-8B models' and 'LLM judge' appear without plain-language glosses. The draft also omits the abstract's premise that an LLM judge is an AI grader used as the reward when no answer key exists. It drops the authors' key stake that this category loss is one 'accuracy and abstention rates do not reveal'. It also omits that two of four judges at answer-key leniency already cost 38-42% of these capitals. Smallest repair: add the premise sentence, gloss the terms (or add term tooltips), and restore the hidden-loss point.

> LLM judges serve as rewards where no answer key exists, in the hope that their knowledge passes to the trained model.

date — pass: originalPublished 2026-11-18 matches the arXiv v1 submission date. The feed date of 2026-11-20 is correctly not used.

> \[Submitted on 18 Nov 2026\]

support — pass: Each claim matches the abstract and is attributed. One wording is slightly firmer than the source. 'did not improve the model's choices' could note the abstract's figure that grader knowledge adds about one corrected answer per hundred questions, but this does not contradict the 'no better' finding. Hedging 'could be memorised' is softer than the source but not misleading. The question and limitation follow from the evidence without unsupported impacts.

> the grader's knowledge adds about one corrected answer per hundred questions

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/3f4d12d5b5f30c952a69921b31676c4264ce176cf30548e99d003ad8e838b4d0/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/blind-spots-travel-knowledge-stays-home-what-rl-inherits-from-3f4d12d5.json)
