{"id":"a56306efbf5dd583ff3d90545d43a0c4c41acefe0cbeff399ff0a3735f6ce3a6","title":"arXiv cs.AI: The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","argument":"The authors introduce The Prestige, a video understanding benchmark built around professional magic performances. Deliberate misdirection challenges models to track events and explain unexpected outcomes. Its open-ended questions cover temporal and causal understanding and robustness against adversarial questions. The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs. They also report systematic differences between human and automated grading, and recurring model failure modes. In ablations, models sometimes select the correct multiple-choice answer without independently generating a correct explanation. The authors say this motivates further investigation into question formats. They present their findings as highlighting limitations in VLM reasoning and its evaluation, and as underscoring the need for more reliable video understanding models and more rigorous benchmarks. The abstract does not quantify the performance gap or describe the benchmark’s size, limiting assessment of the findings’ scope from this account alone.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not quantify the reported performance gap, so its magnitude cannot be assessed from this evidence. The abstract does not state the benchmark’s size or provide details of the evaluated models and procedures, limiting assessment of the findings’ scope.","speaker":{"name":"Wen, Shuo, Du, Beixi, Meriaux, Edwin, Shi, Junming, Si, Chloe, Meger, David, Precup, Doina, Dudek, Gregory","roleAtSource":"Article author"},"source":{"title":"The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.38103","publishedAt":"2026-10-28","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce The Prestige, a video understanding benchmark based on professional magic performances and deliberate misdirection.","excerpt":"We introduce The Prestige, a video understanding benchmark that probes cognitive visual reasoning through professional magic performances, where deliberate misdirection challenges models to track events and explain unexpected outcomes.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"},{"text":"The benchmark includes open-ended questions about temporal and causal understanding and robustness against adversarial questions.","excerpt":"The benchmark comprises open-ended questions ranging from temporal and causal understanding to robustness against adversarial questions.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"},{"text":"The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs.","excerpt":"Evaluation of leading open-source and proprietary VLMs reveals a substantial gap between even the strongest model and human performance.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"},{"text":"The authors report cases where models choose the correct multiple-choice answer but cannot independently generate the correct explanation, and say this motivates further investigation of question formats.","excerpt":"models can select correct multiple-choice answers even when they fail to generate correct explanations independently, motivating further investigations into question formats.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38103","checkedAt":"2026-11-11T09:58:08.225Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do the results change across question formats, given the reported cases where models choose the correct multiple-choice answer but fail to generate a correct explanation?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","summary":"The authors introduce The Prestige, a video understanding benchmark built around professional magic performances. Deliberate misdirection challenges models to track events and explain unexpected outcomes. Its open-ended questions cover temporal and causal understanding and robustness against adversarial questions. The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs. They also report systematic differences between human and automated grading, and recurring model failure modes. In ablations, models sometimes select the correct multiple-choice answer without independently generating a correct explanation. The authors say this motivates further investigation into question formats. They present their findings as highlighting limitations in VLM reasoning and its evaluation, and as underscoring the need for more reliable video understanding models and more rigorous benchmarks. The abstract does not quantify the performance gap or describe the benchmark’s size, limiting assessment of the findings’ scope from this account alone.","question":"How do the results change across question formats, given the reported cases where models choose the correct multiple-choice answer but fail to generate a correct explanation?","claims":[{"text":"The authors introduce The Prestige, a video understanding benchmark based on professional magic performances and deliberate misdirection.","excerpt":"We introduce The Prestige, a video understanding benchmark that probes cognitive visual reasoning through professional magic performances, where deliberate misdirection challenges models to track events and explain unexpected outcomes.","support":"source-stated"},{"text":"The benchmark includes open-ended questions about temporal and causal understanding and robustness against adversarial questions.","excerpt":"The benchmark comprises open-ended questions ranging from temporal and causal understanding to robustness against adversarial questions.","support":"source-stated"},{"text":"The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs.","excerpt":"Evaluation of leading open-source and proprietary VLMs reveals a substantial gap between even the strongest model and human performance.","support":"source-stated"},{"text":"The authors report cases where models choose the correct multiple-choice answer but cannot independently generate the correct explanation, and say this motivates further investigation of question formats.","excerpt":"models can select correct multiple-choice answers even when they fail to generate correct explanations independently, motivating further investigations into question formats.","support":"source-stated"}],"limitations":["The abstract does not quantify the reported performance gap, so its magnitude cannot be assessed from this evidence.","The abstract does not state the benchmark’s size or provide details of the evaluated models and procedures, limiting assessment of the findings’ scope."],"mediaId":"0b3bf28def633ee5b7d2a42ee737100293da9da3a6208d06465dc7e9ec1747a1","correctionReason":null,"id":"a56306efbf5dd583ff3d90545d43a0c4c41acefe0cbeff399ff0a3735f6ce3a6","publicationId":"158907f6-40d1-43ec-8c95-5a9534943271","version":1,"sourceUrl":"https://arxiv.org/abs/2610.38103","sourceTitle":"The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-28","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-30T06:42:45.572Z","createdAt":"2026-11-11T08:55:15.002Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"0b3bf28def633ee5b7d2a42ee737100293da9da3a6208d06465dc7e9ec1747a1","kind":"article","url":"https://arxiv.org/abs/2610.38103","sourceUrl":"https://arxiv.org/abs/2610.38103","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-31T09:01:25.400Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38103","checkedAt":"2026-11-11T09:58:08.225Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"0b3bf28def633ee5b7d2a42ee737100293da9da3a6208d06465dc7e9ec1747a1","confidence":0.73,"provider":"typesafe","model":"jev-1.13.0","inputHash":"2ff72aff9cd03ae7ac0fc60fc6d172c157aec6cf6ccd4ddc8bb7ca989a9f0f08","decidedAt":"2026-10-31T09:01:26.068Z","redecided":{"from":"stream-admission-v5","under":"stream-admission-v5+confidence-0.5"}},"provenance":{"textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observationId":"cc24929adc14354a28fc2662ee8e78572e6a0b1450dc9c1a68386ebc454dccdb","retrievedAt":"2026-10-31T09:01:25.316Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"24be19f2-a52a-4174-834f-0feac69a0d4f"},"reviewer":{"model":"gpt-6-luna","runId":"0eb2db1a-bbde-49a9-8bab-8e77a81ce983"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T08:55:15.002Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wen, Shuo","url":null,"kind":"unknown"},{"name":"Du, Beixi","url":null,"kind":"unknown"},{"name":"Meriaux, Edwin","url":null,"kind":"unknown"},{"name":"Shi, Junming","url":null,"kind":"unknown"},{"name":"Si, Chloe","url":null,"kind":"unknown"},{"name":"Meger, David","url":null,"kind":"unknown"},{"name":"Precup, Doina","url":null,"kind":"unknown"},{"name":"Dudek, Gregory","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wen, Shuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wen, Shuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Du, Beixi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Du, Beixi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Meriaux, Edwin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Meriaux, Edwin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Shi, Junming","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Shi, Junming","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Si, Chloe","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Si, Chloe","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Meger, David","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Meger, David","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Precup, Doina","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Precup, Doina","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Dudek, Gregory","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Dudek, Gregory","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.38103","snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/a56306efbf5dd583ff3d90545d43a0c4c41acefe0cbeff399ff0a3735f6ce3a6/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-the-prestige-benchmarking-cognitive-visual-a56306ef","formats":{"html":"/readings/arxiv-cs-ai-the-prestige-benchmarking-cognitive-visual-a56306ef","markdown":"/readings/arxiv-cs-ai-the-prestige-benchmarking-cognitive-visual-a56306ef.md","json":"/readings/arxiv-cs-ai-the-prestige-benchmarking-cognitive-visual-a56306ef.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"a56306efbf5dd583ff3d90545d43a0c4c41acefe0cbeff399ff0a3735f6ce3a6","revision":"158907f6-40d1-43ec-8c95-5a9534943271","revisionKind":"publication-id","title":"arXiv cs.AI: The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","url":"https://escapemeridian.com/readings/arxiv-cs-ai-the-prestige-benchmarking-cognitive-visual-a56306ef"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.38103","revision":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","revisionKind":"sha256-source-text","title":"The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","url":"https://arxiv.org/abs/2610.38103"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"a56306efbf5dd583ff3d90545d43a0c4c41acefe0cbeff399ff0a3735f6ce3a6","title":"arXiv cs.AI: The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","argument":"The authors introduce The Prestige, a video understanding benchmark built around professional magic performances. Deliberate misdirection challenges models to track events and explain unexpected outcomes. Its open-ended questions cover temporal and causal understanding and robustness against adversarial questions. The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs. They also report systematic differences between human and automated grading, and recurring model failure modes. In ablations, models sometimes select the correct multiple-choice answer without independently generating a correct explanation. The authors say this motivates further investigation into question formats. They present their findings as highlighting limitations in VLM reasoning and its evaluation, and as underscoring the need for more reliable video understanding models and more rigorous benchmarks. The abstract does not quantify the performance gap or describe the benchmark’s size, limiting assessment of the findings’ scope from this account alone.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not quantify the reported performance gap, so its magnitude cannot be assessed from this evidence. The abstract does not state the benchmark’s size or provide details of the evaluated models and procedures, limiting assessment of the findings’ scope.","speaker":{"name":"Wen, Shuo, Du, Beixi, Meriaux, Edwin, Shi, Junming, Si, Chloe, Meger, David, Precup, Doina, Dudek, Gregory","roleAtSource":"Article author"},"source":{"title":"The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.38103","publishedAt":"2026-10-28","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors introduce The Prestige, a video understanding benchmark based on professional magic performances and deliberate misdirection.","excerpt":"We introduce The Prestige, a video understanding benchmark that probes cognitive visual reasoning through professional magic performances, where deliberate misdirection challenges models to track events and explain unexpected outcomes.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"},{"text":"The benchmark includes open-ended questions about temporal and causal understanding and robustness against adversarial questions.","excerpt":"The benchmark comprises open-ended questions ranging from temporal and causal understanding to robustness against adversarial questions.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"},{"text":"The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs.","excerpt":"Evaluation of leading open-source and proprietary VLMs reveals a substantial gap between even the strongest model and human performance.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"},{"text":"The authors report cases where models choose the correct multiple-choice answer but cannot independently generate the correct explanation, and say this motivates further investigation of question formats.","excerpt":"models can select correct multiple-choice answers even when they fail to generate correct explanations independently, motivating further investigations into question formats.","url":"https://arxiv.org/abs/2610.38103","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38103","checkedAt":"2026-11-11T09:58:08.225Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do the results change across question formats, given the reported cases where models choose the correct multiple-choice answer but fail to generate a correct explanation?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","summary":"The authors introduce The Prestige, a video understanding benchmark built around professional magic performances. Deliberate misdirection challenges models to track events and explain unexpected outcomes. Its open-ended questions cover temporal and causal understanding and robustness against adversarial questions. The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs. They also report systematic differences between human and automated grading, and recurring model failure modes. In ablations, models sometimes select the correct multiple-choice answer without independently generating a correct explanation. The authors say this motivates further investigation into question formats. They present their findings as highlighting limitations in VLM reasoning and its evaluation, and as underscoring the need for more reliable video understanding models and more rigorous benchmarks. The abstract does not quantify the performance gap or describe the benchmark’s size, limiting assessment of the findings’ scope from this account alone.","question":"How do the results change across question formats, given the reported cases where models choose the correct multiple-choice answer but fail to generate a correct explanation?","claims":[{"text":"The authors introduce The Prestige, a video understanding benchmark based on professional magic performances and deliberate misdirection.","excerpt":"We introduce The Prestige, a video understanding benchmark that probes cognitive visual reasoning through professional magic performances, where deliberate misdirection challenges models to track events and explain unexpected outcomes.","support":"source-stated"},{"text":"The benchmark includes open-ended questions about temporal and causal understanding and robustness against adversarial questions.","excerpt":"The benchmark comprises open-ended questions ranging from temporal and causal understanding to robustness against adversarial questions.","support":"source-stated"},{"text":"The authors report a substantial gap between human performance and even the strongest evaluated open-source and proprietary VLMs.","excerpt":"Evaluation of leading open-source and proprietary VLMs reveals a substantial gap between even the strongest model and human performance.","support":"source-stated"},{"text":"The authors report cases where models choose the correct multiple-choice answer but cannot independently generate the correct explanation, and say this motivates further investigation of question formats.","excerpt":"models can select correct multiple-choice answers even when they fail to generate correct explanations independently, motivating further investigations into question formats.","support":"source-stated"}],"limitations":["The abstract does not quantify the reported performance gap, so its magnitude cannot be assessed from this evidence.","The abstract does not state the benchmark’s size or provide details of the evaluated models and procedures, limiting assessment of the findings’ scope."],"mediaId":"0b3bf28def633ee5b7d2a42ee737100293da9da3a6208d06465dc7e9ec1747a1","correctionReason":null,"id":"a56306efbf5dd583ff3d90545d43a0c4c41acefe0cbeff399ff0a3735f6ce3a6","publicationId":"158907f6-40d1-43ec-8c95-5a9534943271","version":1,"sourceUrl":"https://arxiv.org/abs/2610.38103","sourceTitle":"The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-28","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-10-30T06:42:45.572Z","createdAt":"2026-11-11T08:55:15.002Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"0b3bf28def633ee5b7d2a42ee737100293da9da3a6208d06465dc7e9ec1747a1","kind":"article","url":"https://arxiv.org/abs/2610.38103","sourceUrl":"https://arxiv.org/abs/2610.38103","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-10-31T09:01:25.400Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.38103","checkedAt":"2026-11-11T09:58:08.225Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"0b3bf28def633ee5b7d2a42ee737100293da9da3a6208d06465dc7e9ec1747a1","confidence":0.73,"provider":"typesafe","model":"jev-1.13.0","inputHash":"2ff72aff9cd03ae7ac0fc60fc6d172c157aec6cf6ccd4ddc8bb7ca989a9f0f08","decidedAt":"2026-10-31T09:01:26.068Z","redecided":{"from":"stream-admission-v5","under":"stream-admission-v5+confidence-0.5"}},"provenance":{"textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observationId":"cc24929adc14354a28fc2662ee8e78572e6a0b1450dc9c1a68386ebc454dccdb","retrievedAt":"2026-10-31T09:01:25.316Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"24be19f2-a52a-4174-834f-0feac69a0d4f"},"reviewer":{"model":"gpt-6-luna","runId":"0eb2db1a-bbde-49a9-8bab-8e77a81ce983"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-11T08:55:15.002Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Wen, Shuo","url":null,"kind":"unknown"},{"name":"Du, Beixi","url":null,"kind":"unknown"},{"name":"Meriaux, Edwin","url":null,"kind":"unknown"},{"name":"Shi, Junming","url":null,"kind":"unknown"},{"name":"Si, Chloe","url":null,"kind":"unknown"},{"name":"Meger, David","url":null,"kind":"unknown"},{"name":"Precup, Doina","url":null,"kind":"unknown"},{"name":"Dudek, Gregory","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Wen, Shuo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Wen, Shuo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Du, Beixi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Du, Beixi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Meriaux, Edwin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Meriaux, Edwin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Shi, Junming","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Shi, Junming","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Si, Chloe","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Si, Chloe","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Meger, David","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Meger, David","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Precup, Doina","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Precup, Doina","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}},{"actor":{"name":"Dudek, Gregory","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Dudek, Gregory","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.38103","locator":"head/meta[name=author|citation_author]"},"assertionId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-11T09:58:08.225Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","sourceUrl":"https://arxiv.org/abs/2610.38103","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z"}}],"participationCoverage":{"total":8,"returned":8,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.38103","snapshotId":"f2d4fd93e67fbd1a1e97497d5d3ac24276232d3991f5dfbe14d47884c38e235a","contentSha256":"105035f65c4f94dedfd71bd96c9155181730bc9e86c9aeba9a91adcd16c0e60e","textHash":"7e8361e4ed8830f9194cc0d13740706df3d364287c4743dffa46abb7690336f4","observedAt":"2026-11-11T09:58:08.225Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/a56306efbf5dd583ff3d90545d43a0c4c41acefe0cbeff399ff0a3735f6ce3a6/source"}}}],"audits":[{"publicationId":"158907f6-40d1-43ec-8c95-5a9534943271","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading consistently attributes the benchmark, results and conclusions to the paper's authors, and the source is correctly identified as the arXiv preprint. The title prefix uses the feed label cs.AI while the primary listing is cs.CV. This is minor and does not misattribute the work.","excerpt":"Authors:Shuo Wen, Beixi Du, Edwin Meriaux, Junming Shi, Chloe Si, David Meger, Doina Precup, Gregory Dudek"},{"dimension":"summary","status":"uncertain","explanation":"The summary faithfully tracks the abstract, but its central term VLMs is never expanded or explained. A general reader also gets no plain-language gloss of ablations or adversarial questions. Smallest repair: write out vision-language models (AI systems that interpret images or video alongside text), or add term-library tooltips for VLM, ablation and adversarial questions. The rest of the content and the question are fine.","excerpt":"Evaluation of leading open-source and proprietary VLMs reveals a substantial gap between even the strongest model and human performance."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-10-28 matches the v1 submission date. The later feed date of Oct 31 is correctly not used.","excerpt":"[Submitted on 28 Oct 2026]"},{"dimension":"support","status":"pass","explanation":"Every claim is directly stated in the abstract and kept as the authors' report. The reading notes that the abstract does not quantify the gap or give the benchmark's size. The question follows from the reported multiple-choice versus explanation discrepancy. No unsupported benefits or costs are implied.","excerpt":"models can select correct multiple-choice answers even when they fail to generate correct explanations independently, motivating further investigations into question formats."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original arXiv abstract page, which is the most direct access to the source.","excerpt":null}],"createdAt":"2026-11-11T08:55:15.002Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}