Ostrow, Mitchell, Qian, Daoyuan, Liu, Toni, Fiete, Ila · Oct 22, 2026
Mitchell Ostrow and coauthors’ mathematical theory of the Jacobian lens
Markdown · JSON · Original source
Mitchell Ostrow, Daoyuan Qian, Toni Liu, and Ila Fiete present a mathematical account of the Jacobian lens (J-lens), an tool for examining representations in language models. They argue that it preferentially detects coherent features: directions in hidden states whose effects amplify across layers, stay roughly constant across token positions, and recur across contexts. The authors say this account explains earlier empirical findings, but they remain agnostic about whether the detected properties are associated with a “Global Workspace.” They test their analysis on a Bayes filter and a toy transformer, both set up to predict next tokens from a Hidden Markov Model. In the Bayes filter, changing task parameters that affect feature coherence also predictably changes the J-lens response. The paper also introduces ω-J-lenses, a family designed to emphasize different temporal scales of information processing in language models. The results offer a proposed explanation for what the J-lens detects; the abstract does not establish how well the account applies beyond the tested systems. How well do these findings generalize to other language models and tasks?
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
q-bio.NC · Article
Source details
Original source date: Oct 22, 2026 (2026-10-22).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-06T02:01:36.894Z. First observed: 2026-10-26T04:01:49.724Z. Retrieved: 2026-10-29T21:17:04.703Z.
Referenced by
Recorded citations and public file attachments. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The abstract describes validation on a Bayes filter and a toy transformer; it does not report validation across a broad range of language models or tasks. The authors explicitly remain agnostic about whether the features identified by the J-lens are associated with a Global Workspace. The evidence here is the paper abstract, so it does not provide the full mathematical derivation or details needed to assess the experiments.
Open question
How well do the authors’ findings generalize beyond the Bayes filter and toy transformer they tested?
Source-stated
The authors present a mathematical theory in which the Jacobian lens preferentially identifies coherent features.
we introduce a mathematical theory of the J-lens, and show that it preferentially identifies coherent features
Source-stated
The paper characterizes those features by effects that amplify across layers, remain roughly constant across token positions, and recur across contexts.
Small perturbations along these directions amplify across layers, remain roughly constant across token positions, and are consistent across different contexts.
Source-stated
The authors validate their analysis on a Bayes filter and a toy transformer that predict next tokens from a Hidden Markov Model.
We validate our analysis on a Bayes filter and a toy transformer, each optimally predicting next tokens emitted from a Hidden Markov Model (HMM).
Source-stated
In the Bayes filter, the authors report that changes to task parameters affecting feature coherence predictably change the J-lens response.
modulating task parameters that change the coherence of features in the Bayes filter correspondingly and predictively modulates the response of the J-lens
Source-stated
The authors do not claim that the described properties necessarily indicate a Global Workspace.
While we are agnostic to whether these properties are necessarily associated with a ``Global Workspace''
Source-stated
The paper introduces a family of J-lenses intended to emphasize different temporal scales of information processing in language models.
a family of J-lenses built to emphasize different temporal scales of information processing in LLMs, which we call $\omega$-J-lenses.
Source and contributor context
Ostrow, Mitchell, Qian, Daoyuan, Liu, Toni, Fiete, Ila · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Interpretability
Research into how a model reaches its outputs, often by examining its internal workings.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 7ec6d681-81b3-4b1c-8705-de908493b8e5.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-06T02:01:36.894Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 1b1f7900f11626fe824782e40d0ed4c2aa44ae29e93b3369e2c99f88dd0a48e8. Source text hash: 42164fb242ed1be17071051cb4d9684312be1964bba3330ed820daf2277c6ebc.
Revision and source-check history
Version 1 · 2026-11-06T02:01:36.894Z
Mitchell Ostrow and coauthors’ mathematical theory of the Jacobian lens
Mitchell Ostrow, Daoyuan Qian, Toni Liu, and Ila Fiete present a mathematical account of the Jacobian lens (J-lens), an tool for examining representations in language models. They argue that it preferentially detects coherent features: directions in hidden states whose effects amplify across layers, stay roughly constant across token positions, and recur across contexts. The authors say this account explains earlier empirical findings, but they remain agnostic about whether the detected properties are associated with a “Global Workspace.” They test their analysis on a Bayes filter and a toy transformer, both set up to predict next tokens from a Hidden Markov Model. In the Bayes filter, changing task parameters that affect feature coherence also predictably changes the J-lens response. The paper also introduces ω-J-lenses, a family designed to emphasize different temporal scales of information processing in language models. The results offer a proposed explanation for what the J-lens detects; the abstract does not establish how well the account applies beyond the tested systems. How well do these findings generalize to other language models and tasks?
Original source
q-bio.NC · Article
Source details
Original source date: Oct 22, 2026 (2026-10-22).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-06T02:01:36.894Z. First observed: 2026-10-26T04:01:49.724Z. Retrieved: 2026-10-29T21:17:04.703Z.
Limits
The abstract describes validation on a Bayes filter and a toy transformer; it does not report validation across a broad range of language models or tasks. The authors explicitly remain agnostic about whether the features identified by the J-lens are associated with a Global Workspace. The evidence here is the paper abstract, so it does not provide the full mathematical derivation or details needed to assess the experiments.
Open question
How well do the authors’ findings generalize beyond the Bayes filter and toy transformer they tested?
Source-stated
The authors present a mathematical theory in which the Jacobian lens preferentially identifies coherent features.
we introduce a mathematical theory of the J-lens, and show that it preferentially identifies coherent features
Source-stated
The paper characterizes those features by effects that amplify across layers, remain roughly constant across token positions, and recur across contexts.
Small perturbations along these directions amplify across layers, remain roughly constant across token positions, and are consistent across different contexts.
Source-stated
The authors validate their analysis on a Bayes filter and a toy transformer that predict next tokens from a Hidden Markov Model.
We validate our analysis on a Bayes filter and a toy transformer, each optimally predicting next tokens emitted from a Hidden Markov Model (HMM).
Source-stated
In the Bayes filter, the authors report that changes to task parameters affecting feature coherence predictably change the J-lens response.
modulating task parameters that change the coherence of features in the Bayes filter correspondingly and predictively modulates the response of the J-lens
Source-stated
The authors do not claim that the described properties necessarily indicate a Global Workspace.
While we are agnostic to whether these properties are necessarily associated with a ``Global Workspace''
Source-stated
The paper introduces a family of J-lenses intended to emphasize different temporal scales of information processing in language models.
a family of J-lenses built to emphasize different temporal scales of information processing in LLMs, which we call $\omega$-J-lenses.
Source and contributor context
Ostrow, Mitchell, Qian, Daoyuan, Liu, Toni, Fiete, Ila · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
Interpretability
Research into how a model reaches its outputs, often by examining its internal workings.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: 7ec6d681-81b3-4b1c-8705-de908493b8e5.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-06T02:01:36.894Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: useful_perspective.
Observation: 1b1f7900f11626fe824782e40d0ed4c2aa44ae29e93b3369e2c99f88dd0a48e8. Source text hash: 42164fb242ed1be17071051cb4d9684312be1964bba3330ed820daf2277c6ebc.
uncertain · 2026-11-06T02:01:36.894Z · anthropic/claude-opus-5.5
Publication: 7ec6d681-81b3-4b1c-8705-de908493b8e5
attribution — pass: The draft correctly credits Mitchell Ostrow, Daoyuan Qian, Toni Liu and Ila Fiete as authors and keeps the claims attributed to them. It does not mention that the J-lens itself came from Anthropic work (Gurnee et al., 2026). That is a context gap rather than a misattribution.
Authors:Mitchell Ostrow, Daoyuan Qian, Toni Liu, Ila Fiete
summary — uncertain: The reading drops essential context: the J-lens is Anthropic's tool, and it found directions in frontier models resembling working-memory-like representations. That finding is why this theory matters. Central terms are not explained in plain language, including interpretability, hidden states, Bayes filter, Hidden Markov Model and Global Workspace. A general reader cannot easily tell why the paper is significant. Smallest repair: add one sentence on the J-lens origin and its working-memory finding, plus short gloss tooltips. The summary also repeats the question field.
This method identified linear directions in the hidden states of frontier language models that had properties reminiscent of working memory-like representations.
date — pass: originalPublished 2026-10-22 matches the arXiv v1 submission date. The later feed date is correctly not used.
[Submitted on 22 Oct 2026]
support — pass: Each claim and limitation is backed by an exact abstract excerpt. The paraphrases of amplification, constancy across positions and consistency across contexts are faithful. The generalization question and limitations are appropriately conditional and do not overstate the toy-system validation.
We validate our analysis on a Bayes filter and a toy transformer, each optimally predicting next tokens emitted from a Hidden Markov Model (HMM).
media — pass: The selected media is the verified arXiv abstract page for this paper. It is the direct original source.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
Research into how a model reaches its outputs, often by examining its internal workings.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivResearch into how a model reaches its outputs, often by examining its internal workings.
Definition referencearXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXiv