{"id":"e17cff6b99ab583ac081248a617c443475d567894054efda470a4c7be11616f5","title":"Eunice Yiu and co-authors argue in a Google DeepMind paper that visual analogies challenge models","argument":"Dated March 5, 2025, the paper by Eunice Yiu and co-authors compares large multimodal models (LMMs) with children and adults on visual analogical reasoning. A visual analogy is a rule inferred from one image and applied to another. The authors propose a benchmark of everyday-object transformations that tests whether systems can identify what changed, describe how it changed, and apply the rule to new scenarios.\n\nThe authors report that the tested models identify what changed effectively but struggle to quantify how it changed and extend the rule to new objects. Children and adults show stronger analogical reasoning at all three stages. GPT-o1, the strongest tested model, does better on simple attributes such as color and size; number, rotation, and reflection are more challenging. The authors say the findings highlight limitations of training models on data primarily consisting of 2D images and text. The abstract does not provide the detailed evaluation methodology or results broken down by task.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"The abstract does not provide detailed evaluation methodology or results broken down by task. Text as retrieved on November 2, 2026; originally published March 5, 2025.","speaker":{"name":"Google DeepMind","roleAtSource":"Publisher"},"source":{"title":"KiVA: Kid-Inspired Visual Analogies for Testing Large Multimodal Models","publisher":"Google DeepMind","format":"Article","url":"https://deepmind.google/research/publications/166018/","publishedAt":"2025-03-05T00:00:00+00:00","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"The paper is dated March 5, 2025.","excerpt":"March 5, 2025","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors propose a benchmark of visual transformations of everyday objects, evaluated in three stages.","excerpt":"We structure the evaluation into three stages: identifying what changed (e.g., color, number, etc.), how it changed (e.g., added one object), and applying the rule to new scenarios.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors report that the tested models identified what changed effectively but struggled to quantify how it changed and extrapolate the rule to new objects.","excerpt":"Our findings show that while GPT-o1, GPT-4V, LLaVA-1.5, and MANTIS identify the \"what\" effectively, they struggle with quantifying the \"how\" and extrapolating this rule to new objects.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors report that children and adults showed stronger analogical reasoning than the tested models at all three stages.","excerpt":"In contrast, children and adults exhibit much stronger analogical reasoning at all three stages.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors report that GPT-o1 did better on simple visual attributes, while number, rotation, and reflection were more challenging.","excerpt":"Additionally, the strongest tested model, GPT-o1, performs better in tasks involving simple surface-level visual attributes like color and size, correlating with quicker human adult response times. Conversely, more complex tasks such as number, rotation, and reflection, which necessitate extensive","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors say the findings highlight limitations of training models primarily on 2D images and text.","excerpt":"Altogether, these findings highlight the limitations of training models on data that primarily consists of 2D images and text.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/166018/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would performance change with training data that better represents spatial properties of the physical world?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Eunice Yiu and co-authors argue in a Google DeepMind paper that visual analogies challenge models","summary":"Dated March 5, 2025, the paper by Eunice Yiu and co-authors compares large multimodal models (LMMs) with children and adults on visual analogical reasoning. A visual analogy is a rule inferred from one image and applied to another. The authors propose a benchmark of everyday-object transformations that tests whether systems can identify what changed, describe how it changed, and apply the rule to new scenarios.\n\nThe authors report that the tested models identify what changed effectively but struggle to quantify how it changed and extend the rule to new objects. Children and adults show stronger analogical reasoning at all three stages. GPT-o1, the strongest tested model, does better on simple attributes such as color and size; number, rotation, and reflection are more challenging. The authors say the findings highlight limitations of training models on data primarily consisting of 2D images and text. The abstract does not provide the detailed evaluation methodology or results broken down by task.","question":"How would performance change with training data that better represents spatial properties of the physical world?","claims":[{"text":"The paper is dated March 5, 2025.","excerpt":"March 5, 2025","support":"source-stated"},{"text":"The authors propose a benchmark of visual transformations of everyday objects, evaluated in three stages.","excerpt":"We structure the evaluation into three stages: identifying what changed (e.g., color, number, etc.), how it changed (e.g., added one object), and applying the rule to new scenarios.","support":"source-stated"},{"text":"The authors report that the tested models identified what changed effectively but struggled to quantify how it changed and extrapolate the rule to new objects.","excerpt":"Our findings show that while GPT-o1, GPT-4V, LLaVA-1.5, and MANTIS identify the \"what\" effectively, they struggle with quantifying the \"how\" and extrapolating this rule to new objects.","support":"source-stated"},{"text":"The authors report that children and adults showed stronger analogical reasoning than the tested models at all three stages.","excerpt":"In contrast, children and adults exhibit much stronger analogical reasoning at all three stages.","support":"source-stated"},{"text":"The authors report that GPT-o1 did better on simple visual attributes, while number, rotation, and reflection were more challenging.","excerpt":"Additionally, the strongest tested model, GPT-o1, performs better in tasks involving simple surface-level visual attributes like color and size, correlating with quicker human adult response times. Conversely, more complex tasks such as number, rotation, and reflection, which necessitate extensive","support":"source-stated"},{"text":"The authors say the findings highlight limitations of training models primarily on 2D images and text.","excerpt":"Altogether, these findings highlight the limitations of training models on data that primarily consists of 2D images and text.","support":"source-stated"}],"limitations":["The abstract does not provide detailed evaluation methodology or results broken down by task.","Text as retrieved on November 2, 2026; originally published March 5, 2025."],"mediaId":null,"correctionReason":null,"id":"e17cff6b99ab583ac081248a617c443475d567894054efda470a4c7be11616f5","publicationId":"baf83190-08a2-dfd2-c7bb-ee5afa2192aa","version":1,"sourceUrl":"https://deepmind.google/research/publications/166018/","sourceTitle":"KiVA: Kid-Inspired Visual Analogies for Testing Large Multimodal Models","publisher":"Google DeepMind","publishedAt":"2025-03-05T00:00:00+00:00","originalPublished":{"value":"2025-03-05T00:00:00+00:00","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-11-02T17:43:51.296Z","createdAt":"2026-11-15T02:33:23.524Z","historical":true,"entityIds":["google-deepmind"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/166018/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"403db0442b2c34ce0a5a562366ccd5bdf8bd6b62fb948b7882a458f05bdff25e","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-11-02T17:43:48.238Z"},"provenance":{"textHash":"6154b828fae2fb4fa4cd04f49fb3b6936a22852081f22a7468d9e46722373c9d","observationId":"7b058ed5b63eaa65367d17ed75da4d3e133d86128b5fee7a7514ff4980d6c33f","retrievedAt":"2026-11-02T17:45:21.612Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"7df580db-6314-828c-c5dc-b6961a5b435e"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"719ec69d-a12c-b691-cb01-8a3015dae2d6"}},"audit":{"status":"passed","reviewedAt":"2026-11-15T02:28:45.107Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/eunice-yiu-and-co-authors-argue-in-a-google-deepmind-paper-that-e17cff6b","formats":{"html":"/readings/eunice-yiu-and-co-authors-argue-in-a-google-deepmind-paper-that-e17cff6b","markdown":"/readings/eunice-yiu-and-co-authors-argue-in-a-google-deepmind-paper-that-e17cff6b.md","json":"/readings/eunice-yiu-and-co-authors-argue-in-a-google-deepmind-paper-that-e17cff6b.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-multimodal-model","name":"Multimodal model","explanation":"A model that works with more than one kind of input or output, such as text, images and audio.","referenceUrl":"https://developers.google.com/machine-learning/glossary#multimodal-model"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"google-deepmind","name":"Google DeepMind","kind":"organization","context":"Google DeepMind develops AI systems for general tasks and scientific research, including AlphaFold.","referenceUrls":["https://deepmind.google/blog/","https://deepmind.google/science/alphafold/"],"path":"/entities/google-deepmind","json":"/entities/google-deepmind.json","markdown":"/entities/google-deepmind.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"e17cff6b99ab583ac081248a617c443475d567894054efda470a4c7be11616f5","revision":"baf83190-08a2-dfd2-c7bb-ee5afa2192aa","revisionKind":"publication-id","title":"Eunice Yiu and co-authors argue in a Google DeepMind paper that visual analogies challenge models","url":"https://escapemeridian.com/readings/eunice-yiu-and-co-authors-argue-in-a-google-deepmind-paper-that-e17cff6b"},"source":{"type":"source","id":"https://deepmind.google/research/publications/166018/","revision":"6154b828fae2fb4fa4cd04f49fb3b6936a22852081f22a7468d9e46722373c9d","revisionKind":"sha256-source-text","title":"KiVA: Kid-Inspired Visual Analogies for Testing Large Multimodal Models","url":"https://deepmind.google/research/publications/166018/"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"e17cff6b99ab583ac081248a617c443475d567894054efda470a4c7be11616f5","title":"Eunice Yiu and co-authors argue in a Google DeepMind paper that visual analogies challenge models","argument":"Dated March 5, 2025, the paper by Eunice Yiu and co-authors compares large multimodal models (LMMs) with children and adults on visual analogical reasoning. A visual analogy is a rule inferred from one image and applied to another. The authors propose a benchmark of everyday-object transformations that tests whether systems can identify what changed, describe how it changed, and apply the rule to new scenarios.\n\nThe authors report that the tested models identify what changed effectively but struggle to quantify how it changed and extend the rule to new objects. Children and adults show stronger analogical reasoning at all three stages. GPT-o1, the strongest tested model, does better on simple attributes such as color and size; number, rotation, and reflection are more challenging. The authors say the findings highlight limitations of training models on data primarily consisting of 2D images and text. The abstract does not provide the detailed evaluation methodology or results broken down by task.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"The abstract does not provide detailed evaluation methodology or results broken down by task. Text as retrieved on November 2, 2026; originally published March 5, 2025.","speaker":{"name":"Google DeepMind","roleAtSource":"Publisher"},"source":{"title":"KiVA: Kid-Inspired Visual Analogies for Testing Large Multimodal Models","publisher":"Google DeepMind","format":"Article","url":"https://deepmind.google/research/publications/166018/","publishedAt":"2025-03-05T00:00:00+00:00","dateBasis":"Date stated on the publisher’s page."},"evidence":[{"text":"The paper is dated March 5, 2025.","excerpt":"March 5, 2025","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors propose a benchmark of visual transformations of everyday objects, evaluated in three stages.","excerpt":"We structure the evaluation into three stages: identifying what changed (e.g., color, number, etc.), how it changed (e.g., added one object), and applying the rule to new scenarios.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors report that the tested models identified what changed effectively but struggled to quantify how it changed and extrapolate the rule to new objects.","excerpt":"Our findings show that while GPT-o1, GPT-4V, LLaVA-1.5, and MANTIS identify the \"what\" effectively, they struggle with quantifying the \"how\" and extrapolating this rule to new objects.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors report that children and adults showed stronger analogical reasoning than the tested models at all three stages.","excerpt":"In contrast, children and adults exhibit much stronger analogical reasoning at all three stages.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors report that GPT-o1 did better on simple visual attributes, while number, rotation, and reflection were more challenging.","excerpt":"Additionally, the strongest tested model, GPT-o1, performs better in tasks involving simple surface-level visual attributes like color and size, correlating with quicker human adult response times. Conversely, more complex tasks such as number, rotation, and reflection, which necessitate extensive","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"},{"text":"The authors say the findings highlight limitations of training models primarily on 2D images and text.","excerpt":"Altogether, these findings highlight the limitations of training models on data that primarily consists of 2D images and text.","url":"https://deepmind.google/research/publications/166018/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/166018/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How would performance change with training data that better represents spatial properties of the physical world?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Eunice Yiu and co-authors argue in a Google DeepMind paper that visual analogies challenge models","summary":"Dated March 5, 2025, the paper by Eunice Yiu and co-authors compares large multimodal models (LMMs) with children and adults on visual analogical reasoning. A visual analogy is a rule inferred from one image and applied to another. The authors propose a benchmark of everyday-object transformations that tests whether systems can identify what changed, describe how it changed, and apply the rule to new scenarios.\n\nThe authors report that the tested models identify what changed effectively but struggle to quantify how it changed and extend the rule to new objects. Children and adults show stronger analogical reasoning at all three stages. GPT-o1, the strongest tested model, does better on simple attributes such as color and size; number, rotation, and reflection are more challenging. The authors say the findings highlight limitations of training models on data primarily consisting of 2D images and text. The abstract does not provide the detailed evaluation methodology or results broken down by task.","question":"How would performance change with training data that better represents spatial properties of the physical world?","claims":[{"text":"The paper is dated March 5, 2025.","excerpt":"March 5, 2025","support":"source-stated"},{"text":"The authors propose a benchmark of visual transformations of everyday objects, evaluated in three stages.","excerpt":"We structure the evaluation into three stages: identifying what changed (e.g., color, number, etc.), how it changed (e.g., added one object), and applying the rule to new scenarios.","support":"source-stated"},{"text":"The authors report that the tested models identified what changed effectively but struggled to quantify how it changed and extrapolate the rule to new objects.","excerpt":"Our findings show that while GPT-o1, GPT-4V, LLaVA-1.5, and MANTIS identify the \"what\" effectively, they struggle with quantifying the \"how\" and extrapolating this rule to new objects.","support":"source-stated"},{"text":"The authors report that children and adults showed stronger analogical reasoning than the tested models at all three stages.","excerpt":"In contrast, children and adults exhibit much stronger analogical reasoning at all three stages.","support":"source-stated"},{"text":"The authors report that GPT-o1 did better on simple visual attributes, while number, rotation, and reflection were more challenging.","excerpt":"Additionally, the strongest tested model, GPT-o1, performs better in tasks involving simple surface-level visual attributes like color and size, correlating with quicker human adult response times. Conversely, more complex tasks such as number, rotation, and reflection, which necessitate extensive","support":"source-stated"},{"text":"The authors say the findings highlight limitations of training models primarily on 2D images and text.","excerpt":"Altogether, these findings highlight the limitations of training models on data that primarily consists of 2D images and text.","support":"source-stated"}],"limitations":["The abstract does not provide detailed evaluation methodology or results broken down by task.","Text as retrieved on November 2, 2026; originally published March 5, 2025."],"mediaId":null,"correctionReason":null,"id":"e17cff6b99ab583ac081248a617c443475d567894054efda470a4c7be11616f5","publicationId":"baf83190-08a2-dfd2-c7bb-ee5afa2192aa","version":1,"sourceUrl":"https://deepmind.google/research/publications/166018/","sourceTitle":"KiVA: Kid-Inspired Visual Analogies for Testing Large Multimodal Models","publisher":"Google DeepMind","publishedAt":"2025-03-05T00:00:00+00:00","originalPublished":{"value":"2025-03-05T00:00:00+00:00","precision":"instant","basis":"backfill:article_published"},"firstObservedAt":"2026-11-02T17:43:51.296Z","createdAt":"2026-11-15T02:33:23.524Z","historical":true,"entityIds":["google-deepmind"],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://deepmind.google/research/publications/166018/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"403db0442b2c34ce0a5a562366ccd5bdf8bd6b62fb948b7882a458f05bdff25e","rubricVersion":"unrecorded","provider":"anthropic","model":"claude-opus-5-5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-11-02T17:43:48.238Z"},"provenance":{"textHash":"6154b828fae2fb4fa4cd04f49fb3b6936a22852081f22a7468d9e46722373c9d","observationId":"7b058ed5b63eaa65367d17ed75da4d3e133d86128b5fee7a7514ff4980d6c33f","retrievedAt":"2026-11-02T17:45:21.612Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"7df580db-6314-828c-c5dc-b6961a5b435e"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"719ec69d-a12c-b691-cb01-8a3015dae2d6"}},"audit":{"status":"passed","reviewedAt":"2026-11-15T02:28:45.107Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"baf83190-08a2-dfd2-c7bb-ee5afa2192aa","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes the paper to Eunice Yiu and co-authors, matching the listed authors. It is hosted on the Google DeepMind publications page, so calling it a Google DeepMind paper reflects the publisher. Findings are consistently framed as what the authors report or say.","excerpt":"Eunice Yiu, Maan Qraitem, Anisa Noor Majhi, Charlie Wong, Yutong Bai, Shiry Ginosar, Alison Gopnik, Kate Saenko"},{"dimension":"summary","status":"pass","explanation":"The summary explains visual analogy in plain language and describes the three-stage evaluation. It reports the comparison with children and adults, the task-type differences and the authors' conclusion about 2D image and text training data. The definitions panel covers multimodal model and benchmark. The title and question fit the source, and its limitation about missing per-task results is accurate.","excerpt":"A \"visual analogy\" is an abstract rule inferred from one image and applied to another."},{"dimension":"date","status":"pass","explanation":"The stated date of March 5, 2025 matches the page dateline and evidence.originalPublished at day precision. Nothing after that date is used, and there is no hindsight framing. The page was modified on 2025-12-10, but the reading relies only on the dated abstract, which is plainly part of the original publication entry. The kind is publisher-page, so no archive disclosure is needed.","excerpt":"March 5, 2025"},{"dimension":"support","status":"pass","explanation":"Each claim is directly supported by abstract text and attributed to the authors. The question about training data that better represents physical spatial properties follows from the authors' stated limitation without asserting an outcome. No benefits or costs are invented. The cited abstract text is plainly original despite the later page edit.","excerpt":"Altogether, these findings highlight the limitations of training models on data that primarily consists of 2D images and text."},{"dimension":"media","status":"pass","explanation":"No media is selected, and no verified media candidates were supplied, so nothing is misidentified or irrelevant.","excerpt":null}],"createdAt":"2026-11-15T02:28:45.107Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}