{"id":"b8c6b71c7e80487c009ec0390989f8928c2f69b8aabc3374f057ad3a22ca4e26","title":"Devin Coldewey argues in TechCrunch that visual AI can miss basic image relationships","argument":"TechCrunch published Devin Coldewey’s article on July 11, 2024. Coldewey argues that visual AI models can fail at simple image tasks, despite marketing language about “vision capabilities” and “visual understanding.” In tests by researchers at Auburn University and the University of Alberta, models struggled to judge whether shapes overlapped and to count interlocking circles. GPT-4o answered correctly only 18% of the time when circles were touching or close together; Gemini 1.5 Pro got close-distance cases right seven times out of ten. Coldewey suggests that familiar images in training data, such as the Olympic Rings, may explain why models succeed on some images and fail on others. Co-author Anh Nguyen cautions that “blind” is an imperfect term and says there is no technology to visualize exactly what a model is seeing. Coldewey’s point is not that visual AI is useless: he says models may be highly accurate on human actions, expressions and everyday scenes, while still lacking the kind of visual understanding these basic tests probe. The study’s simple tasks do not establish how models perform across all image interpretation, and the article does not explain how the models’ performance on those tasks relates to their accuracy on everyday scenes.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 24, 2026; originally published July 11, 2024. The article describes the study as “informal in some ways, but systematic” and does not provide its full methods or results in the retrieved text. The article does not give evidence establishing how performance on the simple tasks relates to accuracy on everyday scenes.","speaker":{"name":"TechCrunch","roleAtSource":"Publisher"},"source":{"title":"'Visual' AI models might not see anything at all","publisher":"TechCrunch","format":"Article","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","publishedAt":"2024-07-11","dateBasis":"Date from the publisher’s archive: its article address or archive listing."},"evidence":[{"text":"Coldewey argues that models’ failures on simple visual tasks challenge the idea that their image processing corresponds to human-like seeing.","excerpt":"The point of this experiment is simply to show that, whatever these models are doing, it doesn’t really correspond with what we think of as seeing.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"The researchers tested multimodal models on simple tasks involving overlap, counting shapes and identifying a circled letter.","excerpt":"They tested the biggest multimodal models on a series of very simple visual tasks, like asking whether two shapes overlap, or how many pentagons are in a picture, or which letter in a word is circled.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"In the reported overlap test, GPT-4o was correct 18% of the time when circles were touching or close together.","excerpt":"at zero or small distances, it got it right only 18% of the time.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"Coldewey suggests that models’ success with five interlocking circles may reflect familiar training data, including the Olympic Rings.","excerpt":"Because they all have a five-circle image prominently featured in their training data: the Olympic Rings.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"Nguyen says “blind” is an imperfect term and that no technology can visualize exactly what a model is seeing.","excerpt":"Currently, there is no technology to visualize exactly what a model is seeing.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"Coldewey says visual AI may still be highly accurate on human actions, expressions and everyday scenes.","excerpt":"Each of these models is likely going to be highly accurate on things like human actions and expressions, photos of everyday objects and situations, and the like.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well do the models perform on the everyday scenes Coldewey says they are intended to interpret, and how was that performance measured?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Devin Coldewey argues in TechCrunch that visual AI can miss basic image relationships","summary":"TechCrunch published Devin Coldewey’s article on July 11, 2024. Coldewey argues that visual AI models can fail at simple image tasks, despite marketing language about “vision capabilities” and “visual understanding.” In tests by researchers at Auburn University and the University of Alberta, models struggled to judge whether shapes overlapped and to count interlocking circles. GPT-4o answered correctly only 18% of the time when circles were touching or close together; Gemini 1.5 Pro got close-distance cases right seven times out of ten. Coldewey suggests that familiar images in training data, such as the Olympic Rings, may explain why models succeed on some images and fail on others. Co-author Anh Nguyen cautions that “blind” is an imperfect term and says there is no technology to visualize exactly what a model is seeing. Coldewey’s point is not that visual AI is useless: he says models may be highly accurate on human actions, expressions and everyday scenes, while still lacking the kind of visual understanding these basic tests probe. The study’s simple tasks do not establish how models perform across all image interpretation, and the article does not explain how the models’ performance on those tasks relates to their accuracy on everyday scenes.","question":"How well do the models perform on the everyday scenes Coldewey says they are intended to interpret, and how was that performance measured?","claims":[{"text":"Coldewey argues that models’ failures on simple visual tasks challenge the idea that their image processing corresponds to human-like seeing.","excerpt":"The point of this experiment is simply to show that, whatever these models are doing, it doesn’t really correspond with what we think of as seeing.","support":"source-stated"},{"text":"The researchers tested multimodal models on simple tasks involving overlap, counting shapes and identifying a circled letter.","excerpt":"They tested the biggest multimodal models on a series of very simple visual tasks, like asking whether two shapes overlap, or how many pentagons are in a picture, or which letter in a word is circled.","support":"source-stated"},{"text":"In the reported overlap test, GPT-4o was correct 18% of the time when circles were touching or close together.","excerpt":"at zero or small distances, it got it right only 18% of the time.","support":"source-stated"},{"text":"Coldewey suggests that models’ success with five interlocking circles may reflect familiar training data, including the Olympic Rings.","excerpt":"Because they all have a five-circle image prominently featured in their training data: the Olympic Rings.","support":"source-stated"},{"text":"Nguyen says “blind” is an imperfect term and that no technology can visualize exactly what a model is seeing.","excerpt":"Currently, there is no technology to visualize exactly what a model is seeing.","support":"source-stated"},{"text":"Coldewey says visual AI may still be highly accurate on human actions, expressions and everyday scenes.","excerpt":"Each of these models is likely going to be highly accurate on things like human actions and expressions, photos of everyday objects and situations, and the like.","support":"source-stated"}],"limitations":["Text as retrieved on October 24, 2026; originally published July 11, 2024.","The article describes the study as “informal in some ways, but systematic” and does not provide its full methods or results in the retrieved text.","The article does not give evidence establishing how performance on the simple tasks relates to accuracy on everyday scenes."],"mediaId":null,"correctionReason":null,"id":"b8c6b71c7e80487c009ec0390989f8928c2f69b8aabc3374f057ad3a22ca4e26","publicationId":"38cdd559-d7c3-0618-b1fb-897b256b18dc","version":1,"sourceUrl":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","sourceTitle":"'Visual' AI models might not see anything at all","publisher":"TechCrunch","publishedAt":null,"originalPublished":{"value":"2024-07-11","precision":"day","basis":"backfill:publisher_archive_date"},"firstObservedAt":"2026-10-24T02:23:58.114Z","createdAt":"2026-11-18T11:38:11.277Z","historical":true,"entityIds":[],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"1fc9c85d72ade477eab929c0ee83300bf890e10edc3e0b98d82aca42f5ad1855","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-24T02:23:55.352Z"},"provenance":{"textHash":"5c4d805b1ae429fcdadc1df16aebb68a22ba63f584849648508c1c2e92afe55b","observationId":"9e8297048cb683818f907f9a9e0cdefae7b26e0f0c60ffbb51f12a8a03a75e1a","retrievedAt":"2026-10-24T02:25:38.465Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"cdaba08d-9f3f-283e-aef1-dbd6d200defb"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"867bd202-9fe2-2a6e-9788-90a4587b7f26"}},"audit":{"status":"passed","reviewedAt":"2026-11-18T11:35:47.542Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/devin-coldewey-argues-in-techcrunch-that-visual-ai-can-miss-b8c6b71c","formats":{"html":"/readings/devin-coldewey-argues-in-techcrunch-that-visual-ai-can-miss-b8c6b71c","markdown":"/readings/devin-coldewey-argues-in-techcrunch-that-visual-ai-can-miss-b8c6b71c.md","json":"/readings/devin-coldewey-argues-in-techcrunch-that-visual-ai-can-miss-b8c6b71c.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"time","name":"TIME","kind":"organization","context":"TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.","referenceUrls":["https://time.com/"],"path":"/entities/time","json":"/entities/time.json","markdown":"/entities/time.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"b8c6b71c7e80487c009ec0390989f8928c2f69b8aabc3374f057ad3a22ca4e26","revision":"38cdd559-d7c3-0618-b1fb-897b256b18dc","revisionKind":"publication-id","title":"Devin Coldewey argues in TechCrunch that visual AI can miss basic image relationships","url":"https://escapemeridian.com/readings/devin-coldewey-argues-in-techcrunch-that-visual-ai-can-miss-b8c6b71c"},"source":{"type":"source","id":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","revision":"5c4d805b1ae429fcdadc1df16aebb68a22ba63f584849648508c1c2e92afe55b","revisionKind":"sha256-source-text","title":"'Visual' AI models might not see anything at all","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"b8c6b71c7e80487c009ec0390989f8928c2f69b8aabc3374f057ad3a22ca4e26","title":"Devin Coldewey argues in TechCrunch that visual AI can miss basic image relationships","argument":"TechCrunch published Devin Coldewey’s article on July 11, 2024. Coldewey argues that visual AI models can fail at simple image tasks, despite marketing language about “vision capabilities” and “visual understanding.” In tests by researchers at Auburn University and the University of Alberta, models struggled to judge whether shapes overlapped and to count interlocking circles. GPT-4o answered correctly only 18% of the time when circles were touching or close together; Gemini 1.5 Pro got close-distance cases right seven times out of ten. Coldewey suggests that familiar images in training data, such as the Olympic Rings, may explain why models succeed on some images and fail on others. Co-author Anh Nguyen cautions that “blind” is an imperfect term and says there is no technology to visualize exactly what a model is seeing. Coldewey’s point is not that visual AI is useless: he says models may be highly accurate on human actions, expressions and everyday scenes, while still lacking the kind of visual understanding these basic tests probe. The study’s simple tasks do not establish how models perform across all image interpretation, and the article does not explain how the models’ performance on those tasks relates to their accuracy on everyday scenes.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on October 24, 2026; originally published July 11, 2024. The article describes the study as “informal in some ways, but systematic” and does not provide its full methods or results in the retrieved text. The article does not give evidence establishing how performance on the simple tasks relates to accuracy on everyday scenes.","speaker":{"name":"TechCrunch","roleAtSource":"Publisher"},"source":{"title":"'Visual' AI models might not see anything at all","publisher":"TechCrunch","format":"Article","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","publishedAt":"2024-07-11","dateBasis":"Date from the publisher’s archive: its article address or archive listing."},"evidence":[{"text":"Coldewey argues that models’ failures on simple visual tasks challenge the idea that their image processing corresponds to human-like seeing.","excerpt":"The point of this experiment is simply to show that, whatever these models are doing, it doesn’t really correspond with what we think of as seeing.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"The researchers tested multimodal models on simple tasks involving overlap, counting shapes and identifying a circled letter.","excerpt":"They tested the biggest multimodal models on a series of very simple visual tasks, like asking whether two shapes overlap, or how many pentagons are in a picture, or which letter in a word is circled.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"In the reported overlap test, GPT-4o was correct 18% of the time when circles were touching or close together.","excerpt":"at zero or small distances, it got it right only 18% of the time.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"Coldewey suggests that models’ success with five interlocking circles may reflect familiar training data, including the Olympic Rings.","excerpt":"Because they all have a five-circle image prominently featured in their training data: the Olympic Rings.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"Nguyen says “blind” is an imperfect term and that no technology can visualize exactly what a model is seeing.","excerpt":"Currently, there is no technology to visualize exactly what a model is seeing.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"},{"text":"Coldewey says visual AI may still be highly accurate on human actions, expressions and everyday scenes.","excerpt":"Each of these models is likely going to be highly accurate on things like human actions and expressions, photos of everyday objects and situations, and the like.","url":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well do the models perform on the everyday scenes Coldewey says they are intended to interpret, and how was that performance measured?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Devin Coldewey argues in TechCrunch that visual AI can miss basic image relationships","summary":"TechCrunch published Devin Coldewey’s article on July 11, 2024. Coldewey argues that visual AI models can fail at simple image tasks, despite marketing language about “vision capabilities” and “visual understanding.” In tests by researchers at Auburn University and the University of Alberta, models struggled to judge whether shapes overlapped and to count interlocking circles. GPT-4o answered correctly only 18% of the time when circles were touching or close together; Gemini 1.5 Pro got close-distance cases right seven times out of ten. Coldewey suggests that familiar images in training data, such as the Olympic Rings, may explain why models succeed on some images and fail on others. Co-author Anh Nguyen cautions that “blind” is an imperfect term and says there is no technology to visualize exactly what a model is seeing. Coldewey’s point is not that visual AI is useless: he says models may be highly accurate on human actions, expressions and everyday scenes, while still lacking the kind of visual understanding these basic tests probe. The study’s simple tasks do not establish how models perform across all image interpretation, and the article does not explain how the models’ performance on those tasks relates to their accuracy on everyday scenes.","question":"How well do the models perform on the everyday scenes Coldewey says they are intended to interpret, and how was that performance measured?","claims":[{"text":"Coldewey argues that models’ failures on simple visual tasks challenge the idea that their image processing corresponds to human-like seeing.","excerpt":"The point of this experiment is simply to show that, whatever these models are doing, it doesn’t really correspond with what we think of as seeing.","support":"source-stated"},{"text":"The researchers tested multimodal models on simple tasks involving overlap, counting shapes and identifying a circled letter.","excerpt":"They tested the biggest multimodal models on a series of very simple visual tasks, like asking whether two shapes overlap, or how many pentagons are in a picture, or which letter in a word is circled.","support":"source-stated"},{"text":"In the reported overlap test, GPT-4o was correct 18% of the time when circles were touching or close together.","excerpt":"at zero or small distances, it got it right only 18% of the time.","support":"source-stated"},{"text":"Coldewey suggests that models’ success with five interlocking circles may reflect familiar training data, including the Olympic Rings.","excerpt":"Because they all have a five-circle image prominently featured in their training data: the Olympic Rings.","support":"source-stated"},{"text":"Nguyen says “blind” is an imperfect term and that no technology can visualize exactly what a model is seeing.","excerpt":"Currently, there is no technology to visualize exactly what a model is seeing.","support":"source-stated"},{"text":"Coldewey says visual AI may still be highly accurate on human actions, expressions and everyday scenes.","excerpt":"Each of these models is likely going to be highly accurate on things like human actions and expressions, photos of everyday objects and situations, and the like.","support":"source-stated"}],"limitations":["Text as retrieved on October 24, 2026; originally published July 11, 2024.","The article describes the study as “informal in some ways, but systematic” and does not provide its full methods or results in the retrieved text.","The article does not give evidence establishing how performance on the simple tasks relates to accuracy on everyday scenes."],"mediaId":null,"correctionReason":null,"id":"b8c6b71c7e80487c009ec0390989f8928c2f69b8aabc3374f057ad3a22ca4e26","publicationId":"38cdd559-d7c3-0618-b1fb-897b256b18dc","version":1,"sourceUrl":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","sourceTitle":"'Visual' AI models might not see anything at all","publisher":"TechCrunch","publishedAt":null,"originalPublished":{"value":"2024-07-11","precision":"day","basis":"backfill:publisher_archive_date"},"firstObservedAt":"2026-10-24T02:23:58.114Z","createdAt":"2026-11-18T11:38:11.277Z","historical":true,"entityIds":[],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://techcrunch.com/2024/07/11/are-visual-ai-models-actually-blind/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"1fc9c85d72ade477eab929c0ee83300bf890e10edc3e0b98d82aca42f5ad1855","rubricVersion":"unrecorded","provider":"unrecorded","model":"anthropic/claude-opus-5.5","reasonCodes":["bridge_admitted_by_model","second_opinion_categorical"],"confidence":1,"decidedAt":"2026-10-24T02:23:55.352Z"},"provenance":{"textHash":"5c4d805b1ae429fcdadc1df16aebb68a22ba63f584849648508c1c2e92afe55b","observationId":"9e8297048cb683818f907f9a9e0cdefae7b26e0f0c60ffbb51f12a8a03a75e1a","retrievedAt":"2026-10-24T02:25:38.465Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"cdaba08d-9f3f-283e-aef1-dbd6d200defb"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"867bd202-9fe2-2a6e-9788-90a4587b7f26"}},"audit":{"status":"passed","reviewedAt":"2026-11-18T11:35:47.542Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"38cdd559-d7c3-0618-b1fb-897b256b18dc","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The page credits the article to Devin Coldewey at TechCrunch. The study is correctly attributed to researchers at Auburn University and the University of Alberta, and the quoted remarks are correctly attributed to co-author Anh Nguyen. The article's own arguments are kept as Coldewey's.","excerpt":"researchers at Auburn University and the University of Alberta"},{"dimension":"summary","status":"pass","explanation":"The summary leads with Coldewey's argument and gives concrete examples (overlapping shapes, counting rings, the Olympic Rings explanation). It also keeps his caveat that the models are not useless. The training-data explanation is fairly framed as a suggestion, matching the source's \"potential answer.\" The title and question are fair, and a non-specialist can follow the central terms.","excerpt":"One potential answer, of course, is staring us right in the face"},{"dimension":"date","status":"pass","explanation":"The original publication date of 2024-07-11 matches the URL and the day-precision evidence, and the dateModified value is the same day. The Most Popular sidebar contains later stories, but the reading neither uses nor relies on them. The page is a publisher-page, not an archive copy, so no archive disclosure is needed. The reading uses no hindsight framing.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim matches its excerpt. The 18% figure, the Gemini 7/10 figure, the list of tasks, the Olympic Rings hypothesis, Nguyen's comments and the everyday-scenes caveat are all source-stated. The limitations and the question flag what the source does not show, without adding unsupported benefits or costs.","excerpt":"Gemini Pro 1.5 does the best, but still only gets 7/10 at close distances."},{"dimension":"media","status":"pass","explanation":"No media candidates were supplied and none was selected, which is consistent with the evidence.","excerpt":null}],"createdAt":"2026-11-18T11:35:47.542Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}