# arXiv paper: How Do Video Foundation Models Encode Intuitive Physics? [Canonical reading](https://escapemeridian.com/readings/arxiv-paper-how-do-video-foundation-models-encode-intuitive-5eeaf94a) In their paper, Samuele Punzo and coauthors ask whether video foundation models encode a usable sense of physical events, and whether it emerges from large-scale video pretraining. They compare frozen representations from V-JEPA, VideoMAE, and LTX-Video on IntPhys2 and Minimal Video Pairs (MVP), using linear, MLP, and temporal attentive probes. They report that physics-relevant information is generally most accessible in intermediate to late layers, but results vary with the benchmark and readout. On MVP, V-JEPA reaches 95.0% pair consistency with temporal attentive probes; on IntPhys2, VideoMAE reaches 73.9% violation-of-expectation accuracy with those probes. The authors argue that intuitive physics is broadly decodable, but not a single, uniform capability: accessibility, especially temporal grounding, depends more on benchmark and readout than on a simple hierarchy of pretraining objectives or model scale. Escape Meridian interpretation: These benchmark results leave open whether the decoded information supports judgments about physical events beyond the tested tasks. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Jun 8, 2026 (2026-06-08). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T12:03:29.043Z. First observed: 2026-11-02T05:04:49.600Z. Retrieved: 2026-11-03T06:02:16.262Z. - [How Do Video Foundation Models Encode Intuitive Physics? Probing Across Pretraining Paradigms](https://arxiv.org/abs/2606.09646) ## Limits The abstract reports benchmark and probe results on IntPhys2 and Minimal Video Pairs (MVP); it does not establish performance on physical-event judgments beyond these tasks. The source describes information as decodable from frozen representations; the abstract does not establish that the models themselves use it reliably in real-world settings. ## Open question Do the representations that probes decode on IntPhys2 and MVP support judgments about physical events beyond those benchmark tasks? ## Source-stated The authors compare frozen-feature probing of V-JEPA, VideoMAE, and LTX-Video on IntPhys2 and Minimal Video Pairs (MVP), using linear, MLP, and temporal attentive probes. > Using frozen-feature probing on IntPhys2 and Minimal Video Pairs (MVP), we compare predictive joint-embedding models (V-JEPA), masked reconstruction models (VideoMAE), and diffusion-based video generators (LTX-Video). - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source-stated The authors report physics-relevant information is generally weakest in early layers and most accessible at intermediate to late depth. > physics-relevant information is generally weakest in early layers and most accessible at intermediate to late depth. - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source-stated On MVP, V-JEPA achieves 95.0% pair consistency with temporal attentive probes; on IntPhys2, VideoMAE achieves 73.9% violation-of-expectation accuracy with those probes. > On MVP, temporal attentive probes substantially outperform linear readouts, with V-JEPA achieving 95.0% pair consistency. On IntPhys2, linear probes recover more signal, and VideoMAE attains the strongest temporal attentive result at 73.9% violation-of-expectation accuracy. - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source-stated The authors conclude that intuitive physics is broadly decodable, but its accessibility and temporal grounding depend more on the benchmark and readout than on a simple hierarchy of pretraining objectives or model scale. > Together, these results suggest that intuitive-physics is broadly decodable from video foundation models, but it is not represented as a single, uniform capability. Its accessibility, and especially its temporal grounding, depends much more on the benchmark and readout than on a simple hierarchy of - [Supporting source](https://arxiv.org/abs/2606.09646) ## Escape Meridian interpretation The evidence describes results on two benchmarks using probes; whether the decoded information supports judgments beyond those tasks is not established in the abstract. > we study whether pretrained video foundation models encode intuitive-physics information in their frozen representations - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source and contributor context Punzo, Samuele, Caselli, Niccolò, Pantelidis, Ippokratis, Massafra, Francesco, Sardo, Salvatore Lo, Salehi, Mohammadreza · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Foundation model A large model trained on broad data that can be adapted to many different tasks. - [Definition reference](https://developers.google.com/machine-learning/glossary#foundation-model) ## Pre-training A model's initial training on a large dataset, before further adaptation for particular uses. - [Definition reference](https://developers.google.com/machine-learning/glossary#pre-training) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: b6a577e8-273c-4732-85b9-6c56051ce866. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T12:03:29.043Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: cb90ec4f9954850f4413619735baa30a98942bbd3e974d2f49e54d512d058fb0. Source text hash: 165a22ec331959d86de6d643f621643ef3e26c0efdd91f718fc6892dae1f370c. ## Revision and source-check history ### Version 1 — 2026-11-10T12:03:29.043Z In their paper, Samuele Punzo and coauthors ask whether video foundation models encode a usable sense of physical events, and whether it emerges from large-scale video pretraining. They compare frozen representations from V-JEPA, VideoMAE, and LTX-Video on IntPhys2 and Minimal Video Pairs (MVP), using linear, MLP, and temporal attentive probes. They report that physics-relevant information is generally most accessible in intermediate to late layers, but results vary with the benchmark and readout. On MVP, V-JEPA reaches 95.0% pair consistency with temporal attentive probes; on IntPhys2, VideoMAE reaches 73.9% violation-of-expectation accuracy with those probes. The authors argue that intuitive physics is broadly decodable, but not a single, uniform capability: accessibility, especially temporal grounding, depends more on benchmark and readout than on a simple hierarchy of pretraining objectives or model scale. Escape Meridian interpretation: These benchmark results leave open whether the decoded information supports judgments about physical events beyond the tested tasks. ## Original source arXiv cs.AI · Article Original source date: Jun 8, 2026 (2026-06-08). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-10T12:03:29.043Z. First observed: 2026-11-02T05:04:49.600Z. Retrieved: 2026-11-03T06:02:16.262Z. - [How Do Video Foundation Models Encode Intuitive Physics? Probing Across Pretraining Paradigms](https://arxiv.org/abs/2606.09646) ## Limits The abstract reports benchmark and probe results on IntPhys2 and Minimal Video Pairs (MVP); it does not establish performance on physical-event judgments beyond these tasks. The source describes information as decodable from frozen representations; the abstract does not establish that the models themselves use it reliably in real-world settings. ## Open question Do the representations that probes decode on IntPhys2 and MVP support judgments about physical events beyond those benchmark tasks? ## Source-stated The authors compare frozen-feature probing of V-JEPA, VideoMAE, and LTX-Video on IntPhys2 and Minimal Video Pairs (MVP), using linear, MLP, and temporal attentive probes. > Using frozen-feature probing on IntPhys2 and Minimal Video Pairs (MVP), we compare predictive joint-embedding models (V-JEPA), masked reconstruction models (VideoMAE), and diffusion-based video generators (LTX-Video). - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source-stated The authors report physics-relevant information is generally weakest in early layers and most accessible at intermediate to late depth. > physics-relevant information is generally weakest in early layers and most accessible at intermediate to late depth. - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source-stated On MVP, V-JEPA achieves 95.0% pair consistency with temporal attentive probes; on IntPhys2, VideoMAE achieves 73.9% violation-of-expectation accuracy with those probes. > On MVP, temporal attentive probes substantially outperform linear readouts, with V-JEPA achieving 95.0% pair consistency. On IntPhys2, linear probes recover more signal, and VideoMAE attains the strongest temporal attentive result at 73.9% violation-of-expectation accuracy. - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source-stated The authors conclude that intuitive physics is broadly decodable, but its accessibility and temporal grounding depend more on the benchmark and readout than on a simple hierarchy of pretraining objectives or model scale. > Together, these results suggest that intuitive-physics is broadly decodable from video foundation models, but it is not represented as a single, uniform capability. Its accessibility, and especially its temporal grounding, depends much more on the benchmark and readout than on a simple hierarchy of - [Supporting source](https://arxiv.org/abs/2606.09646) ## Escape Meridian interpretation The evidence describes results on two benchmarks using probes; whether the decoded information supports judgments beyond those tasks is not established in the abstract. > we study whether pretrained video foundation models encode intuitive-physics information in their frozen representations - [Supporting source](https://arxiv.org/abs/2606.09646) ## Source and contributor context Punzo, Samuele, Caselli, Niccolò, Pantelidis, Ippokratis, Massafra, Francesco, Sardo, Salvatore Lo, Salehi, Mohammadreza · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Foundation model A large model trained on broad data that can be adapted to many different tasks. - [Definition reference](https://developers.google.com/machine-learning/glossary#foundation-model) ## Pre-training A model's initial training on a large dataset, before further adaptation for particular uses. - [Definition reference](https://developers.google.com/machine-learning/glossary#pre-training) ## Benchmark A defined set of tests used to compare systems. Results depend on the tasks and measures selected. - [Definition reference](https://arxiv.org/abs/2211.09110) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: b6a577e8-273c-4732-85b9-6c56051ce866. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-10T12:03:29.043Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: cb90ec4f9954850f4413619735baa30a98942bbd3e974d2f49e54d512d058fb0. Source text hash: 165a22ec331959d86de6d643f621643ef3e26c0efdd91f718fc6892dae1f370c. ### uncertain — 2026-11-10T12:03:29.043Z — anthropic/claude-opus-5.5 Publication: b6a577e8-273c-4732-85b9-6c56051ce866 attribution — pass: The draft correctly attributes the paper to Samuele Punzo and coauthors. It keeps findings framed as what the authors report or argue, and labels the closing interpretation as Escape Meridian's. > Authors:Samuele Punzo, Niccolò Caselli, Ippokratis Pantelidis, Francesco Massafra, Salvatore Lo Sardo, Mohammadreza Salehi summary — uncertain: The reading is faithful but hard for a non-specialist to follow. Central terms such as foundation model, frozen representations, probe (linear, MLP, temporal attentive), pair consistency and violation-of-expectation accuracy are not explained in plain language or via tooltips. The reader is never told that probing tests whether information can be read out of a model. The draft also omits a key comparison: LTX consistently underperforms on attentive probes. Smallest repairs: add a one-sentence plain explanation of probing and these metrics (or term-library tooltips), and add the LTX result. > On both benchmarks, attentive probe performance on LTX consistently underperforms compared to the other two pretraining paradigms. date — pass: originalPublished 2026-06-08 matches the v1 submission date. It is not the later v3 revision or the November feed date. > \[Submitted on 8 Jun 2026 (v1), last revised 30 Oct 2026 (this version, v3)\] support — pass: The numeric results, the layer-depth finding and the authors' conclusion all match the abstract. The interpretation about generalization beyond the benchmarks is explicitly labeled and conditional. The limitations accurately note that decodability is not the same as demonstrated real-world use. > On MVP, temporal attentive probes substantially outperform linear readouts, with V-JEPA achieving 95.0% pair consistency. media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/5eeaf94a1df998299ece1354aec4bbff4a22130299990b3bcc5be05658cce0fd/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-paper-how-do-video-foundation-models-encode-intuitive-5eeaf94a.json)