Luo, Mi, Yu, Lijun, Yu, Adams Wei, Cao, Ang, Sun, Baochen, Yang, Ming-Hsuan, Hua, Nan, Hao, Yuan, Dimakis, Alex, Xu, Ya, Chi, Ed H., Wang, Xuezhi, Liu, Yinxiao · Nov 15, 2026
The Flip Test: Video World Models Fail to Learn the World State
Markdown · JSON · Original source
The authors argue that video can produce plausible-looking predictions without tracking state that determines what happens next. Their FlipTest compares predictions for nearly identical frames where a small change—such as a box moving across a table edge—changes the true future. It also checks whether predictions change only when the future changes, and where that change occurs. Across 11 video world models tested on 1,680 initial frames from simulated and real-world tasks, the authors report that every model fell short; the best achieved 18.3% pair accuracy, below the 25% random-guessing baseline. They say the models also changed predictions when the future stayed the same and did not respond more strongly near the boundary. By contrast, a vision-language model responded selectively, while a coding agent that measured the image came close to the boundary. The authors take this contrast to mean that the deciding information is present in the frame but missed by video models. They urge researchers to rethink world modeling as pure video prediction: predicting the next frame, they argue, must also identify the causal states controlling future dynamics. The reported tests concern the tasks and models evaluated; the abstract does not establish how the results transfer to other settings.
Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.
Original source
cs.AI · Article
Source details
Original source date: Nov 15, 2026 (2026-11-15).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-17T08:46:07.961Z. First observed: 2026-11-17T08:28:33.449Z. Retrieved: 2026-11-17T08:33:03.274Z.
Referenced by
Recorded citations in public readings. A reference alone does not establish agreement or influence.
No public references recorded here yet.
Limits
The supplied evidence gives the paper's abstract, not its full methods or results; it does not provide details needed to assess the evaluation beyond the stated tasks, frames, models, and metrics. The reported results concern the 11 video and tasks evaluated; the abstract does not establish whether they generalize to other models or settings.
Open question
Would video show the same failures on tasks and settings beyond those evaluated in FlipTest?
Source-stated
The authors introduce FlipTest to test whether video world models change predictions when small changes in the present change the true future.
When a small change to the present flips the future, such as a box resting a few millimeters before or past a table edge, a model that tracks the state of the world must flip its prediction too.
Source-stated
FlipTest evaluates 11 video world models on 1,680 initial frames from simulated and real-world tasks; all fall short, and the best reports 18.3% pair accuracy, below the stated 25% random-guessing baseline.
All 11 video world models we evaluate fall short. The best averages 18.3% pair accuracy over real and simulated scenes, below the 25% of random guessing.
Source-stated
The authors report that models change predictions as often when the future stays unchanged as when it changes, and that their response stays flat across dense sweeps.
Models flip as often on steps that leave the future unchanged as on steps that change it, and their response stays flat across the sweeps.
Source-stated
On the same frames, the authors report more selective responses from a vision-language model and a coding agent that measures the image.
On the same frames, a vision-language model is selective and traces a near S-shaped curve near the boundary, and a coding agent that measures the image comes close to the true boundary.
Source-stated
The authors argue that next-frame prediction should identify the causal states that control future dynamics, rather than only draw plausible pixels.
Next-frame prediction must do more than draw plausible pixels; it must also identify the causal states that control future dynamics.
Source and contributor context
Luo, Mi, Yu, Lijun, Yu, Adams Wei, Cao, Ang, Sun, Baochen, Yang, Ming-Hsuan, Hua, Nan, Hao, Yuan, Dimakis, Alex, Xu, Ya, Chi, Ed H., Wang, Xuezhi, Liu, Yinxiao · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
World model
An AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
The World
The World is a public radio news program publishing international reporting on politics, society, labor and technology.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b6d03d8f-e3de-4172-8df7-03496313c657.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-17T08:46:07.961Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 41bccd4faac9ea1a9c1c6bb43dcd7b2e8f901f133b5909dcd1d65e0b70332df4. Source text hash: d73d4a8ad770bebf2b9b3887375912c474b3b7fc60500b1e866057fe5dec739e.
Revision and source-check history
Version 1 · 2026-11-17T08:46:07.961Z
The Flip Test: Video World Models Fail to Learn the World State
The authors argue that video can produce plausible-looking predictions without tracking state that determines what happens next. Their FlipTest compares predictions for nearly identical frames where a small change—such as a box moving across a table edge—changes the true future. It also checks whether predictions change only when the future changes, and where that change occurs. Across 11 video world models tested on 1,680 initial frames from simulated and real-world tasks, the authors report that every model fell short; the best achieved 18.3% pair accuracy, below the 25% random-guessing baseline. They say the models also changed predictions when the future stayed the same and did not respond more strongly near the boundary. By contrast, a vision-language model responded selectively, while a coding agent that measured the image came close to the boundary. The authors take this contrast to mean that the deciding information is present in the frame but missed by video models. They urge researchers to rethink world modeling as pure video prediction: predicting the next frame, they argue, must also identify the causal states controlling future dynamics. The reported tests concern the tasks and models evaluated; the abstract does not establish how the results transfer to other settings.
Original source
cs.AI · Article
Source details
Original source date: Nov 15, 2026 (2026-11-15).
Date basis: Date basis not provided.
Added to Escape Meridian: 2026-11-17T08:46:07.961Z. First observed: 2026-11-17T08:28:33.449Z. Retrieved: 2026-11-17T08:33:03.274Z.
Limits
The supplied evidence gives the paper's abstract, not its full methods or results; it does not provide details needed to assess the evaluation beyond the stated tasks, frames, models, and metrics. The reported results concern the 11 video and tasks evaluated; the abstract does not establish whether they generalize to other models or settings.
Open question
Would video show the same failures on tasks and settings beyond those evaluated in FlipTest?
Source-stated
The authors introduce FlipTest to test whether video world models change predictions when small changes in the present change the true future.
When a small change to the present flips the future, such as a box resting a few millimeters before or past a table edge, a model that tracks the state of the world must flip its prediction too.
Source-stated
FlipTest evaluates 11 video world models on 1,680 initial frames from simulated and real-world tasks; all fall short, and the best reports 18.3% pair accuracy, below the stated 25% random-guessing baseline.
All 11 video world models we evaluate fall short. The best averages 18.3% pair accuracy over real and simulated scenes, below the 25% of random guessing.
Source-stated
The authors report that models change predictions as often when the future stays unchanged as when it changes, and that their response stays flat across dense sweeps.
Models flip as often on steps that leave the future unchanged as on steps that change it, and their response stays flat across the sweeps.
Source-stated
On the same frames, the authors report more selective responses from a vision-language model and a coding agent that measures the image.
On the same frames, a vision-language model is selective and traces a near S-shaped curve near the boundary, and a coding agent that measures the image comes close to the true boundary.
Source-stated
The authors argue that next-frame prediction should identify the causal states that control future dynamics, rather than only draw plausible pixels.
Next-frame prediction must do more than draw plausible pixels; it must also identify the causal states that control future dynamics.
Source and contributor context
Luo, Mi, Yu, Lijun, Yu, Adams Wei, Cao, Ang, Sun, Baochen, Yang, Ming-Hsuan, Hua, Nan, Hao, Yuan, Dimakis, Alex, Xu, Ya, Chi, Ed H., Wang, Xuezhi, Liu, Yinxiao · Article author
These are credits stated by the source; they do not establish a profile match or authorship of every passage.
Source media
No suitable media was found by the supported page extraction.
Terms and references
Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.
World model
An AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
The World
The World is a public radio news program publishing international reporting on politics, society, labor and technology.
arXiv
arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Editorial provenance
Version 1. Publication ID: b6d03d8f-e3de-4172-8df7-03496313c657.
Prepared by gpt-6-luna; grounding review by gpt-6-luna.
Source check: passed by anthropic/claude-opus-5.5 on 2026-11-17T08:46:07.961Z. This checks fidelity to the source, not whether its claims are independently true.
Editorial selection: accept; reasons: material_development.
Observation: 41bccd4faac9ea1a9c1c6bb43dcd7b2e8f901f133b5909dcd1d65e0b70332df4. Source text hash: d73d4a8ad770bebf2b9b3887375912c474b3b7fc60500b1e866057fe5dec739e.
passed · 2026-11-17T08:46:07.961Z · anthropic/claude-opus-5.5
Publication: b6d03d8f-e3de-4172-8df7-03496313c657
attribution — pass: The reading consistently attributes the findings and recommendations to the paper's authors, using phrases like 'the authors argue', 'report' and 'urge'. It does not present the claims as established fact or as Escape Meridian's view. The title matches the arXiv listing.
Authors:Mi Luo, Lijun Yu, Adams Wei Yu, Ang Cao, Baochen Sun, Ming-Hsuan Yang, Nan Hua, Yuan Hao, Alex Dimakis, Ya Xu, Ed H. Chi, Xuezhi Wang, Yinxiao Liu
summary — pass: The summary follows the abstract: the minimal pairs, the matched-control and sweep checks, the 11 models, the 1,680 frames, the 18.3% best score against a 25% random baseline, the contrast with the vision-language model and the coding agent, and the authors' call to action. It keeps the authors' forceful conclusion. The opening sentence and the box example convey what the models try to do well enough for a general reader. One optional tooltip improvement: 'video world model' and 'pair accuracy' could be added to the term library. The question addresses a genuine open issue.
Video world models promise to predict the future of a scene from its present. Do current models keep this promise?
date — pass: The originalPublished date of 2026-11-15 matches the arXiv v1 submission date. The feed date of Nov 17 is correctly not used.
[Submitted on 15 Nov 2026]
support — pass: Each claim is backed by a verbatim abstract excerpt. Claim 4 groups the coding agent under 'more selective responses', whereas the source says only that the agent 'comes close to the true boundary'. This is a minor compression, and the summary itself states it accurately. The limitations properly note that only the abstract is available and that generalization is untested. There are no invented benefits, costs or forecasts.
a coding agent that measures the image comes close to the true boundary
media — pass: The selected media is the verified, direct arXiv abstract page for this paper, which is the original source. It is the most useful access point for readers.
Challenge or correct this reading
Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.
An AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
Definition referenceThe World is a public radio news program publishing international reporting on politics, society, labor and technology.
Explore The WorldarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivAn AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
Definition referenceAn AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
Definition referenceAn AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
Definition referenceThe World is a public radio news program publishing international reporting on politics, society, labor and technology.
Explore The WorldarXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.
Explore arXivAn AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
Definition referenceAn AI system's internal model of how an environment behaves, used to predict what happens next or to plan.
Definition reference