Invariance before structure: two modes of emergent compression in representation learning — arXiv

Invariance before structure: two modes of emergent compression in representation learning — arXiv

Hardik Rajpal and Dan Goodman propose Emergent Adaptive Representation Learning (EARL), a framework for distinguishing two kinds of compression in learned representations. They call the first state-space compression: a representation gains predictive power over its own future beyond what individual network units predict. The second, parameter-space compression, describes networks with different micro-level parameters converging to the same representation. The authors formalise both as Fisher sensitivities and introduce ensemble-level information-theoretic measures. In a recurrent reservoir with a trained bottleneck, they report that parameter-space compression rises within the first few epochs of training, partly independently of the task. State-space compression rises about 30 epochs later, only when the network begins learning the task’s temporal structure. The authors interpret this sequence as evidence that invariance to the substrate and task-specific emergent structure develop in distinct phases. The reported results come from this particular setup; the abstract does not establish whether the timing or sequence holds across other models or tasks.

Publication: 3fe22471-14a1-4b20-815c-272b4066d815

attribution — pass: The framework, measures, experiment and interpretation are correctly attributed to authors Hardik Rajpal and Dan Goodman. The reading consistently presents results as what the authors report or suggest.

Authors:Hardik Rajpal, Dan Goodman

summary — uncertain: The reading is faithful but omits the paper's motivating context: representations show striking invariance across biological and artificial networks. Without it, a non-specialist cannot see why compression phases matter, and the closing word 'substrate' has no anchor. Central terms such as learned representation, Fisher sensitivities, recurrent reservoir, trained bottleneck and epochs are not explained and lack tooltip suggestions. Smallest repair: open with this biological/artificial invariance motivation and add term-library entries for these terms. The title and question are otherwise sound.

Learned representations show striking invariance across biological and artificial neural networks, with different substrates and variations driven by drift, heterogeneity and initialisation.

date — pass: The original publication date of 2026-11-05 matches the v1 submission date. The feed date of Nov 6 is later and was correctly not used.

[Submitted on 5 Nov 2026]

support — pass: The claims are verbatim-grounded. 'Interpret this sequence as evidence' slightly firms up the source's 'suggest', but stays attributed and is acceptable. The limitations, single setup and extended abstract status, are supported. The question follows from the evidence without being leading.

In a recurrent reservoir with a trained bottleneck, parameter-space compression rises within the first few epochs of training and partly independently of the task

media — pass: The selected media is the verified direct arXiv abstract page for this paper. It matches the source URL and is the most direct access point.