AlDahoul, Nouar, Zaki, Yasir · Oct 28, 2026
From Leniency to Suspicion: How an Open Post-Training Recipe Changes Whom LLM Fact-Checkers Believe
Markdown · JSON · Original source
Nouar AlDahoul and Yasir Zaki report that post-training created “job-based doubt” in the open Tülu-3 recipe they studied, but not in the other recipes they tested. They define job-based doubt as judging a claim false more often when its poster has a lower-status job. The reported effect was under one percentage point, though the authors say it was statistically reliable, exceeded training drift and replicated on independent claim sets. In -2 32B, the authors say the model did not start noticing the poster’s job more; instead, a lower-status job shifted its judgment toward “false” after post-training, whereas the base model shifted toward “true.” They attribute this change to a single residual-stream direction and say it occurred at supervised . The authors report confirming the mechanism on 400 new claims with pre-specified predictions, while finding no such doubt in seven deployed commercial models. Their results make the training recipe a key part of auditing speaker bias, rather than assuming the effect applies to every model. Which part of the open Tülu-3 recipe produces the reported change?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T19:31:17.555Z. First observed: 2026-10-30T06:42:52.965Z. Retrieved: 2026-10-30T11:18:34.258Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The source is a preprint under review. The reported effect is under one percentage point, and the findings described here concern the tested model families, claims and post-training recipes. The abstract does not identify which component of the open Tülu-3 recipe produces the reported change.
Open question
Which part of the open Tülu-3 recipe produces the reported change?
Source-stated
The authors report job-based doubt after the open Tülu-3 recipe, but not after the other tested post-training recipes.
post-training creates such doubt only with the open Tülu-3 recipe (OLMo-2 32B, Tülu-3 70B), not with Meta's, Alibaba's or Mistral's
Source-stated
The reported effect was under one percentage point and the authors describe it as statistically reliable, exceeding training drift and replicating on independent claim sets.
the effect is under one percentage point but statistically reliable, exceeding training drift and replicating on independent claim sets
Source-stated
For OLMo-2 32B, the authors report that post-training changed how the model interpreted a lower-status poster’s job, not whether it noticed the job.
the model does not start noticing the poster's job more, but starts interpreting it differently
Source-stated
The authors attribute the change to a single residual-stream direction and report that it happens at supervised fine-tuning.
this flip is carried by a single residual-stream direction and happens at supervised fine-tuning.
Source-stated
The authors report confirming the mechanism on 400 new claims and finding no such doubt in seven deployed commercial models.
The mechanism was confirmed on 400 new claims with pre-specified predictions. None of seven deployed commercial models doubts lower-status posters more.
Source and contributor context
AlDahoul, Nouar, Zaki, Yasir · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Olmo
Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Editorial provenance
Version 1. Publication ID: 97792b54-5b75-49d2-897c-e9b1b9bff78d.
Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T19:31:17.555Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 63b6b50e7a808977bafaca82d0616ca150c858c1d21a9c03ab7687f53ac65f9f. Source text hash: 8ad8fedbad9b6fa141be57bf211caf44f49be48124b96987a7f55b51ab4d0977.
Revision and source-check history
Version 1 · 2026-11-10T19:31:17.555Z
From Leniency to Suspicion: How an Open Post-Training Recipe Changes Whom LLM Fact-Checkers Believe
Nouar AlDahoul and Yasir Zaki report that post-training created “job-based doubt” in the open Tülu-3 recipe they studied, but not in the other recipes they tested. They define job-based doubt as judging a claim false more often when its poster has a lower-status job. The reported effect was under one percentage point, though the authors say it was statistically reliable, exceeded training drift and replicated on independent claim sets. In -2 32B, the authors say the model did not start noticing the poster’s job more; instead, a lower-status job shifted its judgment toward “false” after post-training, whereas the base model shifted toward “true.” They attribute this change to a single residual-stream direction and say it occurred at supervised . The authors report confirming the mechanism on 400 new claims with pre-specified predictions, while finding no such doubt in seven deployed commercial models. Their results make the training recipe a key part of auditing speaker bias, rather than assuming the effect applies to every model. Which part of the open Tülu-3 recipe produces the reported change?
Original source
cs.AI · Article
Source details
Original source date: Oct 28, 2026 (2026-10-28).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-10T19:31:17.555Z. First observed: 2026-10-30T06:42:52.965Z. Retrieved: 2026-10-30T11:18:34.258Z.
Limits
The source is a preprint under review. The reported effect is under one percentage point, and the findings described here concern the tested model families, claims and post-training recipes. The abstract does not identify which component of the open Tülu-3 recipe produces the reported change.
Open question
Which part of the open Tülu-3 recipe produces the reported change?
Source-stated
The authors report job-based doubt after the open Tülu-3 recipe, but not after the other tested post-training recipes.
post-training creates such doubt only with the open Tülu-3 recipe (OLMo-2 32B, Tülu-3 70B), not with Meta's, Alibaba's or Mistral's
Source-stated
The reported effect was under one percentage point and the authors describe it as statistically reliable, exceeding training drift and replicating on independent claim sets.
the effect is under one percentage point but statistically reliable, exceeding training drift and replicating on independent claim sets
Source-stated
For OLMo-2 32B, the authors report that post-training changed how the model interpreted a lower-status poster’s job, not whether it noticed the job.
the model does not start noticing the poster's job more, but starts interpreting it differently
Source-stated
The authors attribute the change to a single residual-stream direction and report that it happens at supervised fine-tuning.
this flip is carried by a single residual-stream direction and happens at supervised fine-tuning.
Source-stated
The authors report confirming the mechanism on 400 new claims and finding no such doubt in seven deployed commercial models.
The mechanism was confirmed on 400 new claims with pre-specified predictions. None of seven deployed commercial models doubts lower-status posters more.
Source and contributor context
AlDahoul, Nouar, Zaki, Yasir · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Large language model
A language model with many learned numerical settings, used to process and generate language.
Fine-tuning
Further training of an existing model to adapt it to a particular task or use.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Olmo
Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Editorial provenance
Version 1. Publication ID: 97792b54-5b75-49d2-897c-e9b1b9bff78d.
Prepared by server-excerpt-repair/stream-excerpt-repair-v1 (original draft: gpt-6-luna); grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T19:31:17.555Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 63b6b50e7a808977bafaca82d0616ca150c858c1d21a9c03ab7687f53ac65f9f. Source text hash: 8ad8fedbad9b6fa141be57bf211caf44f49be48124b96987a7f55b51ab4d0977.
uncertain · 2026-11-10T19:31:17.555Z · anthropic/claude-opus-5.5
Publication: 97792b54-5b75-49d2-897c-e9b1b9bff78d
attribution — pass: The findings are credited to the listed authors, Nouar AlDahoul and Yasir Zaki, and are framed as their reports throughout. The repeated 'the authors say/report' wording keeps the source's claims from reading as established fact.
Authors:Nouar AlDahoul, Yasir Zaki
summary — uncertain: The reading tracks the abstract, but a non-specialist may struggle. It never states the motivating context: LLMs increasingly fact-check online content while seeing who posted it. Central terms such as post-training, base model, supervised fine-tuning, residual-stream direction and training drift are left unexplained and need tooltips. Smallest repair: open with the fact-checking context and add term tooltips. The question asks which part of the recipe produces the change, yet the source locates it at supervised fine-tuning, at least in OLMo-2 32B. The question should acknowledge that and narrow its focus, for example to what within supervised fine-tuning causes it.
LLMs increasingly fact-check online content with the poster's identity in view.
date — pass: The originalPublished date of 2026-10-28 matches the v1 submission date shown on the arXiv page. The feed date of Oct 30 is later and was correctly not used.
[Submitted on 28 Oct 2026]
support — uncertain: The individual claims are faithful, and the small effect size and preprint status are preserved. Two problems remain. First, the limitation says the abstract 'does not identify which component' of the recipe produces the change, but the abstract says the flip happens at supervised fine-tuning. That limitation should be narrowed to what within supervised fine-tuning causes it. Second, 'rather than assuming the effect applies to every model' slightly extends the authors' stated call to audit speaker bias per training recipe.
this flip is carried by a single residual-stream direction and happens at supervised fine-tuning
media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most relevant way to reach the source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Olmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Explore OlmoFurther training of an existing model to adapt it to a particular task or use.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivOlmo is Ai2’s family of language models accompanied by research artifacts intended to make development more inspectable.
Explore OlmoFurther training of an existing model to adapt it to a particular task or use.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv