{"id":"0e23aba6d5b94b62514aaafa76df50cd9c214862a08a92099da0d3cd8e5909bd","title":"Grace Huckins argues in MIT Technology Review that puzzles reveal differences between AI and human cognition","argument":"Grace Huckins’s article was published on August 26, 2026. She argues that puzzles offer a useful view of where AI models succeed and fail, and where people still outperform them—not a simple verdict on intelligence. The article describes language models struggling with spatial reasoning and visual puzzles, and sometimes overlooking small changes in familiar logic problems. Models can do better when puzzles are represented as numbers rather than images, while people can fall for intuitive traps that models avoid. Performance also depends on complexity: in one study, models began to falter as certain puzzles grew more demanding. Commentators questioned whether that showed a distinctive limit of model reasoning or the ordinary difficulty of accumulating complexity. Huckins’s conclusion is that each puzzle highlights at least one difference between machine and human cognition. How well those differences predict performance outside puzzle settings remains unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on November 8, 2026; originally published August 26, 2026. The article discusses performance on puzzles and tests; it does not establish how well those results predict performance on tasks outside those settings.","speaker":{"name":"MIT Technology Review","roleAtSource":"Publisher"},"source":{"title":"AI models flub these intelligence tests. Can you fare any better?","publisher":"MIT Technology Review","format":"Article","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","publishedAt":"2026-08-26T09:00:00Z","dateBasis":"Date from the publisher’s own records, through its API."},"evidence":[{"text":"MIT Technology Review argues that puzzles can provide a useful view of AI models’ strengths and weaknesses, including differences between machine and human cognition.","excerpt":"Seeing where models succeed and fail—and where we humans still beat them—can provide a useful window into the technology’s strengths and weaknesses.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article says that models often struggle with spatial reasoning puzzles, despite their ability to analyze visual inputs.","excerpt":"Though today’s language models typically have the ability to analyze visual inputs, they still fail abysmally at these puzzles.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article reports that models may overlook differences in puzzles resembling examples from training and answer from memorized patterns.","excerpt":"When a puzzle closely resembles one a model saw during training, the model may whiz by key differences and respond with what it memorized.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article reports that models’ performance on some puzzle types faltered as their complexity increased, while commentators questioned what that showed about model reasoning.","excerpt":"But only up to a point: As the number of disks or people hits six and higher, the models began to falter.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article also describes human cognitive errors, including quick intuitive answers that fail when a question is read carefully.","excerpt":"For these questions, humans often give knee-jerk answers, whereas models will respond deliberatively.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article concludes that each puzzle highlights at least one difference between machine and human cognition.","excerpt":"Each one highlights at least one way in which machine and human cognition differ.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well do the differences these puzzles expose predict language models’ performance on tasks outside puzzle tests?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Grace Huckins argues in MIT Technology Review that puzzles reveal differences between AI and human cognition","summary":"Grace Huckins’s article was published on August 26, 2026. She argues that puzzles offer a useful view of where AI models succeed and fail, and where people still outperform them—not a simple verdict on intelligence. The article describes language models struggling with spatial reasoning and visual puzzles, and sometimes overlooking small changes in familiar logic problems. Models can do better when puzzles are represented as numbers rather than images, while people can fall for intuitive traps that models avoid. Performance also depends on complexity: in one study, models began to falter as certain puzzles grew more demanding. Commentators questioned whether that showed a distinctive limit of model reasoning or the ordinary difficulty of accumulating complexity. Huckins’s conclusion is that each puzzle highlights at least one difference between machine and human cognition. How well those differences predict performance outside puzzle settings remains unknown.","question":"How well do the differences these puzzles expose predict language models’ performance on tasks outside puzzle tests?","claims":[{"text":"MIT Technology Review argues that puzzles can provide a useful view of AI models’ strengths and weaknesses, including differences between machine and human cognition.","excerpt":"Seeing where models succeed and fail—and where we humans still beat them—can provide a useful window into the technology’s strengths and weaknesses.","support":"source-stated"},{"text":"The article says that models often struggle with spatial reasoning puzzles, despite their ability to analyze visual inputs.","excerpt":"Though today’s language models typically have the ability to analyze visual inputs, they still fail abysmally at these puzzles.","support":"source-stated"},{"text":"The article reports that models may overlook differences in puzzles resembling examples from training and answer from memorized patterns.","excerpt":"When a puzzle closely resembles one a model saw during training, the model may whiz by key differences and respond with what it memorized.","support":"source-stated"},{"text":"The article reports that models’ performance on some puzzle types faltered as their complexity increased, while commentators questioned what that showed about model reasoning.","excerpt":"But only up to a point: As the number of disks or people hits six and higher, the models began to falter.","support":"source-stated"},{"text":"The article also describes human cognitive errors, including quick intuitive answers that fail when a question is read carefully.","excerpt":"For these questions, humans often give knee-jerk answers, whereas models will respond deliberatively.","support":"source-stated"},{"text":"The article concludes that each puzzle highlights at least one difference between machine and human cognition.","excerpt":"Each one highlights at least one way in which machine and human cognition differ.","support":"source-stated"}],"limitations":["Text as retrieved on November 8, 2026; originally published August 26, 2026.","The article discusses performance on puzzles and tests; it does not establish how well those results predict performance on tasks outside those settings."],"mediaId":null,"correctionReason":null,"id":"0e23aba6d5b94b62514aaafa76df50cd9c214862a08a92099da0d3cd8e5909bd","publicationId":"a506cc47-1fb2-888c-cee6-348378d459ff","version":1,"sourceUrl":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","sourceTitle":"AI models flub these intelligence tests. Can you fare any better?","publisher":"MIT Technology Review","publishedAt":"2026-08-26T09:00:00Z","originalPublished":{"value":"2026-08-26T09:00:00Z","precision":"instant","basis":"backfill:api_published"},"firstObservedAt":"2026-11-08T23:47:39.431Z","createdAt":"2026-11-09T03:28:37.785Z","historical":true,"entityIds":[],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"d70662fe57d5fed71124792470bb32feb1cbb40ddce6c8dc8ef042d44ef60ea1","rubricVersion":"unrecorded","provider":"typesafe","model":"jev-1.13.0","reasonCodes":["bridge_admitted_by_model"],"confidence":0.63,"decidedAt":"2026-11-08T23:47:39.263Z"},"provenance":{"textHash":"a92f0a303e6e326fe4c7717b603fc29157c2279ce68b0ef347da989a18c2883a","observationId":"cf902295e6908247191595e69e464b59fd6694f990dc70e598cfdf8b31734f23","retrievedAt":"2026-11-08T23:49:34.592Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"28ea2f8b-3fdc-a833-33fd-e508ba3b4a20"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"2aad154d-d213-ced1-3abd-63957f119096"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T03:23:43.506Z","model":"anthropic/claude-opus-5.5"}},"canonicalUrl":"https://escapemeridian.com/readings/grace-huckins-argues-in-mit-technology-review-that-puzzles-0e23aba6","formats":{"html":"/readings/grace-huckins-argues-in-mit-technology-review-that-puzzles-0e23aba6","markdown":"/readings/grace-huckins-argues-in-mit-technology-review-that-puzzles-0e23aba6.md","json":"/readings/grace-huckins-argues-in-mit-technology-review-that-puzzles-0e23aba6.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"0e23aba6d5b94b62514aaafa76df50cd9c214862a08a92099da0d3cd8e5909bd","revision":"a506cc47-1fb2-888c-cee6-348378d459ff","revisionKind":"publication-id","title":"Grace Huckins argues in MIT Technology Review that puzzles reveal differences between AI and human cognition","url":"https://escapemeridian.com/readings/grace-huckins-argues-in-mit-technology-review-that-puzzles-0e23aba6"},"source":{"type":"source","id":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","revision":"a92f0a303e6e326fe4c7717b603fc29157c2279ce68b0ef347da989a18c2883a","revisionKind":"sha256-source-text","title":"AI models flub these intelligence tests. Can you fare any better?","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"0e23aba6d5b94b62514aaafa76df50cd9c214862a08a92099da0d3cd8e5909bd","title":"Grace Huckins argues in MIT Technology Review that puzzles reveal differences between AI and human cognition","argument":"Grace Huckins’s article was published on August 26, 2026. She argues that puzzles offer a useful view of where AI models succeed and fail, and where people still outperform them—not a simple verdict on intelligence. The article describes language models struggling with spatial reasoning and visual puzzles, and sometimes overlooking small changes in familiar logic problems. Models can do better when puzzles are represented as numbers rather than images, while people can fall for intuitive traps that models avoid. Performance also depends on complexity: in one study, models began to falter as certain puzzles grew more demanding. Commentators questioned whether that showed a distinctive limit of model reasoning or the ordinary difficulty of accumulating complexity. Huckins’s conclusion is that each puzzle highlights at least one difference between machine and human cognition. How well those differences predict performance outside puzzle settings remains unknown.","topicIds":["artificial-general-intelligence"],"domain":"other","claimType":"attributed reading","limits":"Text as retrieved on November 8, 2026; originally published August 26, 2026. The article discusses performance on puzzles and tests; it does not establish how well those results predict performance on tasks outside those settings.","speaker":{"name":"MIT Technology Review","roleAtSource":"Publisher"},"source":{"title":"AI models flub these intelligence tests. Can you fare any better?","publisher":"MIT Technology Review","format":"Article","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","publishedAt":"2026-08-26T09:00:00Z","dateBasis":"Date from the publisher’s own records, through its API."},"evidence":[{"text":"MIT Technology Review argues that puzzles can provide a useful view of AI models’ strengths and weaknesses, including differences between machine and human cognition.","excerpt":"Seeing where models succeed and fail—and where we humans still beat them—can provide a useful window into the technology’s strengths and weaknesses.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article says that models often struggle with spatial reasoning puzzles, despite their ability to analyze visual inputs.","excerpt":"Though today’s language models typically have the ability to analyze visual inputs, they still fail abysmally at these puzzles.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article reports that models may overlook differences in puzzles resembling examples from training and answer from memorized patterns.","excerpt":"When a puzzle closely resembles one a model saw during training, the model may whiz by key differences and respond with what it memorized.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article reports that models’ performance on some puzzle types faltered as their complexity increased, while commentators questioned what that showed about model reasoning.","excerpt":"But only up to a point: As the number of disks or people hits six and higher, the models began to falter.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article also describes human cognitive errors, including quick intuitive answers that fail when a question is read carefully.","excerpt":"For these questions, humans often give knee-jerk answers, whereas models will respond deliberatively.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"},{"text":"The article concludes that each puzzle highlights at least one difference between machine and human cognition.","excerpt":"Each one highlights at least one way in which machine and human cognition differ.","url":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","relationship":"Source-stated"}],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","checkedAt":null,"items":[]},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well do the differences these puzzles expose predict language models’ performance on tasks outside puzzle tests?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Grace Huckins argues in MIT Technology Review that puzzles reveal differences between AI and human cognition","summary":"Grace Huckins’s article was published on August 26, 2026. She argues that puzzles offer a useful view of where AI models succeed and fail, and where people still outperform them—not a simple verdict on intelligence. The article describes language models struggling with spatial reasoning and visual puzzles, and sometimes overlooking small changes in familiar logic problems. Models can do better when puzzles are represented as numbers rather than images, while people can fall for intuitive traps that models avoid. Performance also depends on complexity: in one study, models began to falter as certain puzzles grew more demanding. Commentators questioned whether that showed a distinctive limit of model reasoning or the ordinary difficulty of accumulating complexity. Huckins’s conclusion is that each puzzle highlights at least one difference between machine and human cognition. How well those differences predict performance outside puzzle settings remains unknown.","question":"How well do the differences these puzzles expose predict language models’ performance on tasks outside puzzle tests?","claims":[{"text":"MIT Technology Review argues that puzzles can provide a useful view of AI models’ strengths and weaknesses, including differences between machine and human cognition.","excerpt":"Seeing where models succeed and fail—and where we humans still beat them—can provide a useful window into the technology’s strengths and weaknesses.","support":"source-stated"},{"text":"The article says that models often struggle with spatial reasoning puzzles, despite their ability to analyze visual inputs.","excerpt":"Though today’s language models typically have the ability to analyze visual inputs, they still fail abysmally at these puzzles.","support":"source-stated"},{"text":"The article reports that models may overlook differences in puzzles resembling examples from training and answer from memorized patterns.","excerpt":"When a puzzle closely resembles one a model saw during training, the model may whiz by key differences and respond with what it memorized.","support":"source-stated"},{"text":"The article reports that models’ performance on some puzzle types faltered as their complexity increased, while commentators questioned what that showed about model reasoning.","excerpt":"But only up to a point: As the number of disks or people hits six and higher, the models began to falter.","support":"source-stated"},{"text":"The article also describes human cognitive errors, including quick intuitive answers that fail when a question is read carefully.","excerpt":"For these questions, humans often give knee-jerk answers, whereas models will respond deliberatively.","support":"source-stated"},{"text":"The article concludes that each puzzle highlights at least one difference between machine and human cognition.","excerpt":"Each one highlights at least one way in which machine and human cognition differ.","support":"source-stated"}],"limitations":["Text as retrieved on November 8, 2026; originally published August 26, 2026.","The article discusses performance on puzzles and tests; it does not establish how well those results predict performance on tasks outside those settings."],"mediaId":null,"correctionReason":null,"id":"0e23aba6d5b94b62514aaafa76df50cd9c214862a08a92099da0d3cd8e5909bd","publicationId":"a506cc47-1fb2-888c-cee6-348378d459ff","version":1,"sourceUrl":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","sourceTitle":"AI models flub these intelligence tests. Can you fare any better?","publisher":"MIT Technology Review","publishedAt":"2026-08-26T09:00:00Z","originalPublished":{"value":"2026-08-26T09:00:00Z","precision":"instant","basis":"backfill:api_published"},"firstObservedAt":"2026-11-08T23:47:39.431Z","createdAt":"2026-11-09T03:28:37.785Z","historical":true,"entityIds":[],"topicIds":["artificial-general-intelligence"],"media":[],"sourceMedia":{"status":"unchecked","sourceUrl":"https://www.technologyreview.com/2026/08/26/1141952/puzzles-ai-models-flub-these-tests/","checkedAt":null,"items":[]},"editorialDecision":{"outcome":"accept","mediaId":null,"inputHash":"d70662fe57d5fed71124792470bb32feb1cbb40ddce6c8dc8ef042d44ef60ea1","rubricVersion":"unrecorded","provider":"typesafe","model":"jev-1.13.0","reasonCodes":["bridge_admitted_by_model"],"confidence":0.63,"decidedAt":"2026-11-08T23:47:39.263Z"},"provenance":{"textHash":"a92f0a303e6e326fe4c7717b603fc29157c2279ce68b0ef347da989a18c2883a","observationId":"cf902295e6908247191595e69e464b59fd6694f990dc70e598cfdf8b31734f23","retrievedAt":"2026-11-08T23:49:34.592Z","kind":"publisher-page","preparer":{"model":"openai/gpt-6-luna","runId":"28ea2f8b-3fdc-a833-33fd-e508ba3b4a20"},"reviewer":{"model":"anthropic/claude-opus-5.5","runId":"2aad154d-d213-ced1-3abd-63957f119096"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T03:23:43.506Z","model":"anthropic/claude-opus-5.5"}}}],"audits":[{"publicationId":"a506cc47-1fb2-888c-cee6-348378d459ff","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"This is a single article, an interactive puzzle feature, credited in the text to Grace Huckins of MIT Technology Review. The draft attributes the argument to Huckins and the publication. Claim 1 says 'MIT Technology Review argues'; that is acceptable because the author is the publication's reporter. Studies and commentators are attributed as the article reports them.","excerpt":"Grace Huckins is an AI reporter at MIT Technology Review. They have a PhD in neuroscience."},{"dimension":"summary","status":"uncertain","explanation":"The reading leaves out the article's framing that AI is improving 'a lot—and quickly', for example from 18% to near-perfect on Connections puzzles and gains on ARC-AGI. It also omits the hedge 'at least for now'. Without these, the reading leans toward weakness alone. It labels 'each puzzle highlights' as Huckins's conclusion, but that line comes from the introduction. It also does not say the piece is an interactive quiz for readers. Smallest repair: add one sentence noting the rapid improvement, and reword 'conclusion' as 'framing'.","excerpt":"Judged purely on its puzzling skills, AI is improving a lot—and quickly."},{"dimension":"date","status":"pass","explanation":"The draft gives August 26, 2026, which matches the UTC calendar day of originalPublished (2026-08-26T09:00:00Z). The page was edited on 2026-09-18, but the reading relies only on main-body passages that are plainly part of the original piece. The later 'Deep Dive' sidebar is neither used nor relied on. The reading uses no hindsight framing and presents no forecast as an outcome. The kind is publisher-page, so no archive disclosure is needed.","excerpt":null},{"dimension":"support","status":"pass","explanation":"Each claim's excerpt matches the article body. The summary's statements are supported by the source: better performance with numeric grid encodings, humans' knee-jerk errors, and commentators' doubts about the Apple study. The open question about performance outside puzzle tests is framed as unknown and does not overreach. The page was edited after publication, but all cited passages are core article text.","excerpt":"The Apple paper went viral, but commentators questioned whether the results reveal a unique limitation of LLM reasoning—or just that it’s normal to make errors as complexity piles up."},{"dimension":"media","status":"pass","explanation":"No media was selected, and no verified media candidates were supplied, so no media misidentification is possible.","excerpt":null}],"createdAt":"2026-11-09T03:23:43.506Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}