{"id":"6a56a6f3814fb66761c7fb25194b78eedf0c7e6ff26451747eb330a4d6535b1d","title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","argument":"The authors argue that verifying scientific figures means checking whether each figure supports the claims that cite it—not merely describing a figure or answering questions about one selected in advance. A model must also find what is worth checking and recognize a contradiction. They test this autonomous task on 31 full manuscripts with reviewer-documented figure-text inconsistencies hidden from the models. In open-ended reviews, 60.6% mentioned the flawed figure, but the grading rubric accepted only 3.3% as diagnosing its contradiction. The authors caution that a blind audit found the rubric lenient, so they compare conditions rather than treat these rates as absolute performance. In a controlled run, naming the figure without saying it was flawed raised the accepted diagnosis rate from 4.3% to 20.4%; a consistency instruction and coarse visual cues did not. When a contradiction appeared only in the rendered figure, text-only review produced no accepted diagnosis, while review of rendered pages with the figure named reached 15.8%. A centrality-based baseline for choosing figures reached 4.8%. The authors conclude that deciding what to verify is a large part of the difficulty, and their tested selection rule does not resolve it.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The study tests 31 manuscripts, each containing a reviewer-documented figure-text inconsistency; the abstract does not establish how the results generalize to papers without such inconsistencies. The authors say the grading rubric was lenient and describe accepted rates as comparisons between conditions rather than absolute values.","speaker":{"name":"Zhang, Guijia, Qiu, Hongbo, Zha, Jiajun, Cai, Guohao, Zheng, Hao, Jing, Huihao, Cheng, Kexu, Luo, Shipeng, Huang, JinYang, Cheung, Shing-Chi, Lu, Changsheng, Che, Haoxuan, Yang, Harry","roleAtSource":"Article author"},"source":{"title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.24353","publishedAt":"2026-11-18","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors test autonomous figure-text verification on 31 full manuscripts with reviewer-documented inconsistencies hidden from the models.","excerpt":"We study this autonomous setting on 31 full manuscripts, each carrying a reviewer-documented figure-text inconsistency from PRISMM-Bench that is hidden from the model.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"Most open-ended reviews mentioned the flawed figure, but few received an accepted diagnosis; the authors caution that the rubric was lenient.","excerpt":"60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction; a blind audit finds the rubric lenient","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"Naming the figure without identifying it as flawed raised the rubric-accepted diagnosis rate, while a consistency instruction and coarse visual cues did not.","excerpt":"naming the figure, without stating that it is flawed, raises rubric-accepted diagnosis from 4.3% to 20.4%, while a consistency instruction and coarse visual cues do not.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"The tested centrality-based selection baseline did not resolve the difficulty of deciding what to verify.","excerpt":"A baseline that lets the model choose which figures to check, ranked by centrality, stays at 4.8%. Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"The authors call for evaluating whether multimodal document understanding can propose what to verify, not merely answer questions about a supplied figure.","excerpt":"Multimodal document understanding should be evaluated on whether a model can propose what to verify, not only answer questions about a figure it is handed.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24353","checkedAt":"2026-11-19T06:52:10.363Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What selection rule could let a model identify which figures in a complete paper are worth checking and diagnose contradictions the tested centrality baseline missed?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","summary":"The authors argue that verifying scientific figures means checking whether each figure supports the claims that cite it—not merely describing a figure or answering questions about one selected in advance. A model must also find what is worth checking and recognize a contradiction. They test this autonomous task on 31 full manuscripts with reviewer-documented figure-text inconsistencies hidden from the models. In open-ended reviews, 60.6% mentioned the flawed figure, but the grading rubric accepted only 3.3% as diagnosing its contradiction. The authors caution that a blind audit found the rubric lenient, so they compare conditions rather than treat these rates as absolute performance. In a controlled run, naming the figure without saying it was flawed raised the accepted diagnosis rate from 4.3% to 20.4%; a consistency instruction and coarse visual cues did not. When a contradiction appeared only in the rendered figure, text-only review produced no accepted diagnosis, while review of rendered pages with the figure named reached 15.8%. A centrality-based baseline for choosing figures reached 4.8%. The authors conclude that deciding what to verify is a large part of the difficulty, and their tested selection rule does not resolve it.","question":"What selection rule could let a model identify which figures in a complete paper are worth checking and diagnose contradictions the tested centrality baseline missed?","claims":[{"text":"The authors test autonomous figure-text verification on 31 full manuscripts with reviewer-documented inconsistencies hidden from the models.","excerpt":"We study this autonomous setting on 31 full manuscripts, each carrying a reviewer-documented figure-text inconsistency from PRISMM-Bench that is hidden from the model.","support":"source-stated"},{"text":"Most open-ended reviews mentioned the flawed figure, but few received an accepted diagnosis; the authors caution that the rubric was lenient.","excerpt":"60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction; a blind audit finds the rubric lenient","support":"source-stated"},{"text":"Naming the figure without identifying it as flawed raised the rubric-accepted diagnosis rate, while a consistency instruction and coarse visual cues did not.","excerpt":"naming the figure, without stating that it is flawed, raises rubric-accepted diagnosis from 4.3% to 20.4%, while a consistency instruction and coarse visual cues do not.","support":"source-stated"},{"text":"The tested centrality-based selection baseline did not resolve the difficulty of deciding what to verify.","excerpt":"A baseline that lets the model choose which figures to check, ranked by centrality, stays at 4.8%. Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it.","support":"source-stated"},{"text":"The authors call for evaluating whether multimodal document understanding can propose what to verify, not merely answer questions about a supplied figure.","excerpt":"Multimodal document understanding should be evaluated on whether a model can propose what to verify, not only answer questions about a figure it is handed.","support":"source-stated"}],"limitations":["The study tests 31 manuscripts, each containing a reviewer-documented figure-text inconsistency; the abstract does not establish how the results generalize to papers without such inconsistencies.","The authors say the grading rubric was lenient and describe accepted rates as comparisons between conditions rather than absolute values."],"mediaId":"4da10dfc4e4386ef4e2b548e1661314ef246cbaa3fb46f6b62b613fbea92a34a","correctionReason":null,"id":"6a56a6f3814fb66761c7fb25194b78eedf0c7e6ff26451747eb330a4d6535b1d","publicationId":"905ebcbc-322c-4ebc-81a8-4fdb2ae0f21b","version":1,"sourceUrl":"https://arxiv.org/abs/2611.24353","sourceTitle":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-18","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-19T05:01:27.845Z","createdAt":"2026-11-19T05:49:05.158Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"4da10dfc4e4386ef4e2b548e1661314ef246cbaa3fb46f6b62b613fbea92a34a","kind":"article","url":"https://arxiv.org/abs/2611.24353","sourceUrl":"https://arxiv.org/abs/2611.24353","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-19T05:47:57.688Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24353","checkedAt":"2026-11-19T06:52:10.363Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"4da10dfc4e4386ef4e2b548e1661314ef246cbaa3fb46f6b62b613fbea92a34a","confidence":0.75,"provider":"typesafe","model":"jev-1.13.0","inputHash":"ed58c79420919c1f6d812371dacfc0b75b91ee38b5c0932ef632cbedb2b71e92","decidedAt":"2026-11-19T05:47:59.495Z"},"provenance":{"textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observationId":"27776f4fbca549b12611e9214126911067dea2d3422442644aec415c242c586b","retrievedAt":"2026-11-19T05:47:57.495Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"71029a2d-44c6-45c4-8713-0f58c370aba8"},"reviewer":{"model":"gpt-6-luna","runId":"9664710d-1fe9-4a30-8aa5-14e7c84902e7"}},"audit":{"status":"passed","reviewedAt":"2026-11-19T05:49:05.158Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Zhang, Guijia","url":null,"kind":"unknown"},{"name":"Qiu, Hongbo","url":null,"kind":"unknown"},{"name":"Zha, Jiajun","url":null,"kind":"unknown"},{"name":"Cai, Guohao","url":null,"kind":"unknown"},{"name":"Zheng, Hao","url":null,"kind":"unknown"},{"name":"Jing, Huihao","url":null,"kind":"unknown"},{"name":"Cheng, Kexu","url":null,"kind":"unknown"},{"name":"Luo, Shipeng","url":null,"kind":"unknown"},{"name":"Huang, JinYang","url":null,"kind":"unknown"},{"name":"Cheung, Shing-Chi","url":null,"kind":"unknown"},{"name":"Lu, Changsheng","url":null,"kind":"unknown"},{"name":"Che, Haoxuan","url":null,"kind":"unknown"},{"name":"Yang, Harry","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Zhang, Guijia","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhang, Guijia","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Qiu, Hongbo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qiu, Hongbo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Zha, Jiajun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zha, Jiajun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Cai, Guohao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cai, Guohao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Zheng, Hao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zheng, Hao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Jing, Huihao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jing, Huihao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Cheng, Kexu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheng, Kexu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Luo, Shipeng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luo, Shipeng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Huang, JinYang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Huang, JinYang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:8","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Cheung, Shing-Chi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheung, Shing-Chi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:9","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Lu, Changsheng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lu, Changsheng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:10","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Che, Haoxuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Che, Haoxuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:11","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Yang, Harry","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Harry","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:12","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}}],"participationCoverage":{"total":13,"returned":13,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.24353","snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z","basis":"publisher-page-metadata","sourceLinks":51,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/6a56a6f3814fb66761c7fb25194b78eedf0c7e6ff26451747eb330a4d6535b1d/source"}},"canonicalUrl":"https://escapemeridian.com/readings/when-figures-disagree-with-text-evaluating-autonomous-visual-6a56a6f3","formats":{"html":"/readings/when-figures-disagree-with-text-evaluating-autonomous-visual-6a56a6f3","markdown":"/readings/when-figures-disagree-with-text-evaluating-autonomous-visual-6a56a6f3.md","json":"/readings/when-figures-disagree-with-text-evaluating-autonomous-visual-6a56a6f3.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"conditions":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"},{"id":"figure","name":"Figure","kind":"organization","context":"Figure develops humanoid robots and publishes its own tests of how they perform physical tasks.","referenceUrls":["https://www.figure.ai/news","https://www.figure.ai/news/helix-2-5-zero-shot-30-home-generalization"],"path":"/entities/figure","json":"/entities/figure.json","markdown":"/entities/figure.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"6a56a6f3814fb66761c7fb25194b78eedf0c7e6ff26451747eb330a4d6535b1d","revision":"905ebcbc-322c-4ebc-81a8-4fdb2ae0f21b","revisionKind":"publication-id","title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","url":"https://escapemeridian.com/readings/when-figures-disagree-with-text-evaluating-autonomous-visual-6a56a6f3"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.24353","revision":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","revisionKind":"sha256-source-text","title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","url":"https://arxiv.org/abs/2611.24353"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter."},"revisions":[{"id":"6a56a6f3814fb66761c7fb25194b78eedf0c7e6ff26451747eb330a4d6535b1d","title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","argument":"The authors argue that verifying scientific figures means checking whether each figure supports the claims that cite it—not merely describing a figure or answering questions about one selected in advance. A model must also find what is worth checking and recognize a contradiction. They test this autonomous task on 31 full manuscripts with reviewer-documented figure-text inconsistencies hidden from the models. In open-ended reviews, 60.6% mentioned the flawed figure, but the grading rubric accepted only 3.3% as diagnosing its contradiction. The authors caution that a blind audit found the rubric lenient, so they compare conditions rather than treat these rates as absolute performance. In a controlled run, naming the figure without saying it was flawed raised the accepted diagnosis rate from 4.3% to 20.4%; a consistency instruction and coarse visual cues did not. When a contradiction appeared only in the rendered figure, text-only review produced no accepted diagnosis, while review of rendered pages with the figure named reached 15.8%. A centrality-based baseline for choosing figures reached 4.8%. The authors conclude that deciding what to verify is a large part of the difficulty, and their tested selection rule does not resolve it.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The study tests 31 manuscripts, each containing a reviewer-documented figure-text inconsistency; the abstract does not establish how the results generalize to papers without such inconsistencies. The authors say the grading rubric was lenient and describe accepted rates as comparisons between conditions rather than absolute values.","speaker":{"name":"Zhang, Guijia, Qiu, Hongbo, Zha, Jiajun, Cai, Guohao, Zheng, Hao, Jing, Huihao, Cheng, Kexu, Luo, Shipeng, Huang, JinYang, Cheung, Shing-Chi, Lu, Changsheng, Che, Haoxuan, Yang, Harry","roleAtSource":"Article author"},"source":{"title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.24353","publishedAt":"2026-11-18","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors test autonomous figure-text verification on 31 full manuscripts with reviewer-documented inconsistencies hidden from the models.","excerpt":"We study this autonomous setting on 31 full manuscripts, each carrying a reviewer-documented figure-text inconsistency from PRISMM-Bench that is hidden from the model.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"Most open-ended reviews mentioned the flawed figure, but few received an accepted diagnosis; the authors caution that the rubric was lenient.","excerpt":"60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction; a blind audit finds the rubric lenient","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"Naming the figure without identifying it as flawed raised the rubric-accepted diagnosis rate, while a consistency instruction and coarse visual cues did not.","excerpt":"naming the figure, without stating that it is flawed, raises rubric-accepted diagnosis from 4.3% to 20.4%, while a consistency instruction and coarse visual cues do not.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"The tested centrality-based selection baseline did not resolve the difficulty of deciding what to verify.","excerpt":"A baseline that lets the model choose which figures to check, ranked by centrality, stays at 4.8%. Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"},{"text":"The authors call for evaluating whether multimodal document understanding can propose what to verify, not merely answer questions about a supplied figure.","excerpt":"Multimodal document understanding should be evaluated on whether a model can propose what to verify, not only answer questions about a figure it is handed.","url":"https://arxiv.org/abs/2611.24353","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24353","checkedAt":"2026-11-19T06:52:10.363Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"What selection rule could let a model identify which figures in a complete paper are worth checking and diagnose contradictions the tested centrality baseline missed?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","summary":"The authors argue that verifying scientific figures means checking whether each figure supports the claims that cite it—not merely describing a figure or answering questions about one selected in advance. A model must also find what is worth checking and recognize a contradiction. They test this autonomous task on 31 full manuscripts with reviewer-documented figure-text inconsistencies hidden from the models. In open-ended reviews, 60.6% mentioned the flawed figure, but the grading rubric accepted only 3.3% as diagnosing its contradiction. The authors caution that a blind audit found the rubric lenient, so they compare conditions rather than treat these rates as absolute performance. In a controlled run, naming the figure without saying it was flawed raised the accepted diagnosis rate from 4.3% to 20.4%; a consistency instruction and coarse visual cues did not. When a contradiction appeared only in the rendered figure, text-only review produced no accepted diagnosis, while review of rendered pages with the figure named reached 15.8%. A centrality-based baseline for choosing figures reached 4.8%. The authors conclude that deciding what to verify is a large part of the difficulty, and their tested selection rule does not resolve it.","question":"What selection rule could let a model identify which figures in a complete paper are worth checking and diagnose contradictions the tested centrality baseline missed?","claims":[{"text":"The authors test autonomous figure-text verification on 31 full manuscripts with reviewer-documented inconsistencies hidden from the models.","excerpt":"We study this autonomous setting on 31 full manuscripts, each carrying a reviewer-documented figure-text inconsistency from PRISMM-Bench that is hidden from the model.","support":"source-stated"},{"text":"Most open-ended reviews mentioned the flawed figure, but few received an accepted diagnosis; the authors caution that the rubric was lenient.","excerpt":"60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction; a blind audit finds the rubric lenient","support":"source-stated"},{"text":"Naming the figure without identifying it as flawed raised the rubric-accepted diagnosis rate, while a consistency instruction and coarse visual cues did not.","excerpt":"naming the figure, without stating that it is flawed, raises rubric-accepted diagnosis from 4.3% to 20.4%, while a consistency instruction and coarse visual cues do not.","support":"source-stated"},{"text":"The tested centrality-based selection baseline did not resolve the difficulty of deciding what to verify.","excerpt":"A baseline that lets the model choose which figures to check, ranked by centrality, stays at 4.8%. Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it.","support":"source-stated"},{"text":"The authors call for evaluating whether multimodal document understanding can propose what to verify, not merely answer questions about a supplied figure.","excerpt":"Multimodal document understanding should be evaluated on whether a model can propose what to verify, not only answer questions about a figure it is handed.","support":"source-stated"}],"limitations":["The study tests 31 manuscripts, each containing a reviewer-documented figure-text inconsistency; the abstract does not establish how the results generalize to papers without such inconsistencies.","The authors say the grading rubric was lenient and describe accepted rates as comparisons between conditions rather than absolute values."],"mediaId":"4da10dfc4e4386ef4e2b548e1661314ef246cbaa3fb46f6b62b613fbea92a34a","correctionReason":null,"id":"6a56a6f3814fb66761c7fb25194b78eedf0c7e6ff26451747eb330a4d6535b1d","publicationId":"905ebcbc-322c-4ebc-81a8-4fdb2ae0f21b","version":1,"sourceUrl":"https://arxiv.org/abs/2611.24353","sourceTitle":"When Figures Disagree with Text: Evaluating Autonomous Visual Evidence Verification in Scientific Papers","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-18","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-19T05:01:27.845Z","createdAt":"2026-11-19T05:49:05.158Z","historical":false,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"4da10dfc4e4386ef4e2b548e1661314ef246cbaa3fb46f6b62b613fbea92a34a","kind":"article","url":"https://arxiv.org/abs/2611.24353","sourceUrl":"https://arxiv.org/abs/2611.24353","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-19T05:47:57.688Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.24353","checkedAt":"2026-11-19T06:52:10.363Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v9","mediaId":"4da10dfc4e4386ef4e2b548e1661314ef246cbaa3fb46f6b62b613fbea92a34a","confidence":0.75,"provider":"typesafe","model":"jev-1.13.0","inputHash":"ed58c79420919c1f6d812371dacfc0b75b91ee38b5c0932ef632cbedb2b71e92","decidedAt":"2026-11-19T05:47:59.495Z"},"provenance":{"textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observationId":"27776f4fbca549b12611e9214126911067dea2d3422442644aec415c242c586b","retrievedAt":"2026-11-19T05:47:57.495Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"71029a2d-44c6-45c4-8713-0f58c370aba8"},"reviewer":{"model":"gpt-6-luna","runId":"9664710d-1fe9-4a30-8aa5-14e7c84902e7"}},"audit":{"status":"passed","reviewedAt":"2026-11-19T05:49:05.158Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Zhang, Guijia","url":null,"kind":"unknown"},{"name":"Qiu, Hongbo","url":null,"kind":"unknown"},{"name":"Zha, Jiajun","url":null,"kind":"unknown"},{"name":"Cai, Guohao","url":null,"kind":"unknown"},{"name":"Zheng, Hao","url":null,"kind":"unknown"},{"name":"Jing, Huihao","url":null,"kind":"unknown"},{"name":"Cheng, Kexu","url":null,"kind":"unknown"},{"name":"Luo, Shipeng","url":null,"kind":"unknown"},{"name":"Huang, JinYang","url":null,"kind":"unknown"},{"name":"Cheung, Shing-Chi","url":null,"kind":"unknown"},{"name":"Lu, Changsheng","url":null,"kind":"unknown"},{"name":"Che, Haoxuan","url":null,"kind":"unknown"},{"name":"Yang, Harry","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Zhang, Guijia","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhang, Guijia","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Qiu, Hongbo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Qiu, Hongbo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Zha, Jiajun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zha, Jiajun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Cai, Guohao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cai, Guohao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Zheng, Hao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zheng, Hao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Jing, Huihao","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Jing, Huihao","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Cheng, Kexu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheng, Kexu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Luo, Shipeng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Luo, Shipeng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:7","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Huang, JinYang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Huang, JinYang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:8","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Cheung, Shing-Chi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheung, Shing-Chi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:9","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Lu, Changsheng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Lu, Changsheng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:10","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Che, Haoxuan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Che, Haoxuan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:11","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}},{"actor":{"name":"Yang, Harry","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yang, Harry","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.24353","locator":"head/meta[name=author|citation_author]"},"assertionId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967:participation:12","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-19T06:52:10.363Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","sourceUrl":"https://arxiv.org/abs/2611.24353","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z"}}],"participationCoverage":{"total":13,"returned":13,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.24353","snapshotId":"bfe65f4e2b826ba5d3ba01fe2c60624d36c630e3d25959ca0ad05fdd6c6f5967","contentSha256":"d13241e0248555970fd32af7df7ba8e3093bc98f45b6f979ac3415f522705ad6","textHash":"0299c91ce69b28f669e52cf375350723ac9490d659b197089d7147a991594f11","observedAt":"2026-11-19T06:52:10.363Z","basis":"publisher-page-metadata","sourceLinks":51,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/6a56a6f3814fb66761c7fb25194b78eedf0c7e6ff26451747eb330a4d6535b1d/source"}}}],"audits":[{"publicationId":"905ebcbc-322c-4ebc-81a8-4fdb2ae0f21b","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The arguments, results and conclusions are attributed to the paper's authors on arXiv. The rubric caveat is presented as the authors' own caution, not as an independent finding.","excerpt":"a blind audit finds the rubric lenient, so we read these as rubric-accepted rates and compare conditions rather than absolute values"},{"dimension":"summary","status":"pass","explanation":"The title matches the source. The summary leads with the authors' argument, explains the autonomous verification task in plain terms and reports the key figures accurately: 31 manuscripts, 60.6% vs 3.3%, 4.3% to 20.4%, 15.8% and 4.8%. It keeps the rubric caveat and the authors' conclusion. Omitting that six multimodal models were tested, and not defining centrality, are minor and do not block understanding. The question follows directly from the authors' stated unresolved issue.","excerpt":"Deciding what to verify is thus a large part of the difficulty, and the selection rule we tested does not resolve it."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-18 matches the arXiv v1 submission date. The feed date of Nov 19 is later and correctly not used.","excerpt":"[Submitted on 18 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim's excerpt is a verbatim match and supports the claim. Describing 60.6% as most reviews is accurate. The limitations are faithful to the source: the authors flag the lenient rubric, and the draft notes that generalization beyond papers with documented inconsistencies is unestablished. No benefits, costs or forecasts are invented.","excerpt":"Of 1,445 open-ended reviews, 60.6% mention the flawed figure, yet a grading rubric accepts only 3.3% as diagnosing its contradiction"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original arXiv abstract page. It is the most direct way to access the source.","excerpt":null}],"createdAt":"2026-11-19T05:49:05.158Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}